Abstract
Implanted brain–computer interfaces (iBCIs) have made impressive advances since the first human proof-of-concept demonstrations. Humans with long-standing paralysis have been able to communicate by typing at productive rates and have controlled their own arm or a robotic arm to perform useful reach-and-grasp actions like drinking. Importantly, the body of available evidence so far does not raise safety concerns. However, neuroscience and engineering challenges remain for useful, generally available iBCIs. The two most formidable problems now are, first, a stable, long-lasting implanted electrode and, second, a sufficient understanding of neural information coding principles to generate rich, reliable, and flexible motor commands. Fully implantable microelectronics systems capable of signal processing and wireless transmission, as well as devices for high-throughput generation of command signals, are complex issues but appear to be feasible now or in the near term. Thus the ability to restore productive function, including movement of paralyzed limbs, to millions of people with paralysis at a requisite commercial scale is very promising. Consideration of costs, user needs, and regulatory matters, as well as ethical, legal, and social implications, must also be included as this technology develops.
Keywords
Brain computer interface, Ethics, Functional electrical stimulation, Human, iBCI, Microelectrode, Microelectrode array, Movement, Neural decoding, Paralysis
Acknowledgments
I would like to thank the many students and colleagues who are or have been part of the Brown University team that has laid the groundwork and implemented the translation of brain–computer interfaces, as well as the larger Braingate team, which includes Case Western Reserve University and Stanford, for their energy, dedication, intelligence, and exceptional collaborative spirit (they can be found at www.braingate.org ). I would also like to thank Gabriela Santesso and Jo Bowler of the Wyss Center for their assistance in preparing this chapter. Finally, I would like to thank the editors for their helpful comments in improving this chapter.
Introduction
Beginning in the early 2000s, a series of human pilot trials has demonstrated “proof of concept” of the potential for implanted brain–computer interface (iBCI) systems: People with severe paralysis have been able to control computer cursors and physical devices, using only the activity of a small ensemble of neurons in the cerebral cortex ( ). The advent of iBCIs was built upon a vast base of prior accomplishments. First, fundamental neuroscience in animals provided core scientific knowledge needed to understand where and how movement is represented in the brain. Second, BCI research emerged as a distinct neurotechnology field, built upon fundamental and translational discoveries leading to the design, implementation, and translation of BCIs for human application. Third, computational neuroscience developments provided a mathematical and algorithmic framework to decode neural signals into commands. Fourth, technology advances provided both (1) implantable sensors that could detect useful signals over long times and (2) computational devices in powerful, affordable architectures able to perform decoding in real time.
It has become apparent, beginning with scalp-based EEG demonstrations, that even the simplest BCIs could provide clinical benefit for people, especially those who are severely paralyzed and unable to communicate or care for themselves ( ). Severe paralysis, including the inability to move the body and limbs for useful actions, can result from large strokes (mainly in the brainstem), upper level spinal cord injury (SCI), or neurodegenerative diseases like amyotrophic lateral sclerosis (ALS). BCIs capturing neural signals at high resolution, which requires implanted sensors, have the potential to extend clinical benefits to those with less severe paralysis (e.g., hemiplegia) and to offer a range of control options, including reanimation of paralyzed limbs. In the years leading up to iBCI human proof of concept, several fundamental neuroscience discoveries were particularly important in laying the foundation for eventual clinical translation. 1
1 Regrettably, no single review yet has successfully captured the complex history of essential knowledge and engineering advances that produced the current iBCI field, in part because the success of BCIs draws upon advances in so many fields. Attribution to any one person for key advances is always open to debate, so one must consult multiple sources for a comprehensive perspective of BCI history.
In my view, the work of Ed Evarts, a brilliant neurophysiologist at the National Institute of Mental Health who passed away prematurely, 22 http://www.nasonline.org/search.html?q=evarts&submit.x=0&submit.y=0 .
created one of the most important foundational elements for iBCIs to develop. He showed that the spike rate of single motor cortex neurons in behaving primates coded movement variables (a number in a time window) ( ). The results also reinforced the idea that the motor cortex initiated behavior because activity commenced before movement. Subsequent work that adopted the Evarts method of single-neuron recording in conscious monkeys showed that more complex motor behavior could be decoded from neuron population activity using a simple mathematical framework to combine the activity of small populations of motor cortex neurons. In particular, Apostolos Georgopoulos and his students, and others inspired by the population coding concept, showed that motor cortex population collective activity provided reasonable estimates of the direction of reach or other motor variables ( ). A collection of additional nonhuman primate studies in the early 2000s, probably an inflection point in iBCI development, provided a series of advances that led to: (1) an implantable multielectrode array (MEA) technology suitable for human use that could record ensembles of ∼100 neurons in humans ( ), (2) the demonstration that arm actions could be decoded from roughly randomly selected ensembles of fewer than 100 neurons simultaneously recorded (∼dozens of neurons) from a small patch of cortex ( ), and (3) the BCI proof of concept that able-bodied monkeys could use decoded representations of neural population activity in real time to control a computer cursor ( ) or a robotic arm ( ) to perform behavioral tasks they first learned using their arm. Of the many other contributors to the fundamental background necessary to reach this point, two of particular historical note are (1) the demonstration already in the late 1960s by Eberhard Fetz that monkeys could learn to control a single motor cortex neuron ( ) and (2) the call issued by William Heetderks’ National Institutes of Health (NIH) Neural Prosthesis Program in 1990 ( ). This NIH program requested proposals for researchers to develop a BCI to restore hand function in animals that could eventually help people with SCI regain control ( Fig. 25.1 ). This program and its leaders, and a succession of BCI programs mounted by the Defense Advanced Research Projects Agency ( ), arguably established a framework for, inspired, and set the time frame for the considerable number of researchers who went on to realize this vision in monkeys, and then in humans, and to create the currently flourishing BCI field.
Across the following decade iBCI studies, now involving more than a dozen humans, repeatedly confirmed the ability of people with paralysis stemming from many different origins to control devices. Although not formally consolidated in a single report, none of these studies involving many years of implant evaluation report safety concerns or infection. For iBCIs, control emerges when desired actions are imagined—no learning is required. Thus, neural activity emerging when a person imagines moving his or her arm to a set of spatial locations can be mapped directly to commands to move a robot arm or computer cursor to those locations. People paralyzed from neurodegenerative disorders, brainstem stroke, or SCI have been able to use a few dozen signal channels of spiking-level data from one or sometimes two microelectrode arrays to control cursors, to operate a computer for communication (e.g., typing interface), or to carry out useful everyday reach-and-grasp actions such as drinking or eating, using a multijoint robotic arm ( ). Importantly, the study provided one example in which a sensor implanted 5 years earlier provided useful neural signals, indicating that long-lasting interfaces could be feasible. In the same person, reliable control occurred across each of 5 days around the 1000th day after this implant ( ), affirming the potential for useful control years after an interface has been implanted.
The BCI field has somewhat divided into two areas, one developing communication and the other emphasizing movement applications. These applications require different implementations of assistive technology, even though both rely on having useful control signals. Fig. 25.1 provides a schematic of the range of BCI applications, users, and technology to convey the spectrum of applications and the potential value of many different types of BCIs. Both communication and movement applications of BCIs have made progress. People with severe paralysis who have limited ability to communicate, because they cannot speak or cannot move their hands or arms well enough to type or write, are able to use spelling and other communication devices through a computer ( ). Communication rates have continued to improve so that iBCIs now far exceed rates obtained by EEG signals ( ), achieving around 34 characters/min. While this typing speed is still about one-sixth the rate of an able-bodied typist, it is at least five times the rate achieved by noninvasive BCI systems. A second BCI direction is to re-create arm functions, by using either a robotic arm or the person’s own arm. Perhaps the most ambitious vision, as foreshadowed in the 1990 National Institute of Neurological Disorders and Stroke call ( Fig. 25.1 ), is to reanimate the arm with a physical bridge from the brain to the arm. This vision aims to restore brain-to-body control via a physical connection. This replacement neural system requires not only a BCI that can collect neural signals complex enough to provide flexible control of the enormously complex set of arm muscles ( ), but also functional electrical stimulation (FES) system technology.
The FES system provides coordinated neuromuscular stimulation so that neural signals can produce natural arm movements ( ). This vision is feasible. In 2017 a person with SCI was able to use a neural interface to an implanted FES system to produce multijoint reach-and-grasp actions with that person’s own limb ( ). Thus both rapid communication and reanimation of the body have achieved proof of concept in humans, but not the kind of control achieved by intact biological systems.
Despite the past years of iBCI communication and movement control proof of concept in humans, a commercial product is not available as of this writing, although there have been prior attempts. The BCI field had one early commercial launch attempt with two startup companies: Neural Signals and Cyberkinetics. Cyberkinetics, founded in the early 2000s by the author and colleagues, was a venture capital–funded effort to produce a BCI that could restore movement and control. Although it successfully translated preclinical work from the author’s laboratory to human early-stage (FDA investigational device exemption or IDE) trials of the BrainGate BCI system for people with severe paralysis, it ceased operations in 2009 owing to a lack of funding. The clinical trial was transferred back to the academic group, which now, as part of a consortium, continues to develop BrainGate under an IDE. As will be described below, Cyberkinetics and their academic collaborators provided an important proof of concept in the early 2000s that signals recorded through chronically implanted electrode arrays in paralyzed humans could be used as control signals ( ). The principles and technology established in this trial are used in all ongoing human iBCI trials today. Further, this large step to translate basic research to humans would have taken many additional years without the translational team and the large capital investment that this company was able to attract. Neural Signals was founded in the late 1980s by Philip Kennedy. Although Neural Signals remains in operation, it does not have active clinical trials of an iBCI as of this writing. Neural Signals was a pioneer in showing that neural interfaces could be placed in humans ( ) and also was important in guiding the translation of BCIs to humans. Interestingly, there appears to be a second wave of commercial interest in very early stage iBCI-related companies, Kernel and Neuralink. Both are backed by successful entrepreneurs (Bryan Johnson and Elon Musk) who appear to be aiming to advance BCI development. Even with substantial capital, which is essential to translate such complex medical devices to humans, effective iBCIs still face considerable development challenges that will be characterized in this chapter.
Looking at the present state of the field, existing iBCI systems face complex biological and technological issues: control is slower, more variable, less dexterous, and less accurate than can be achieved by an able-bodied person; the technology used is bulky, cumbersome, and not portable; the implant requires a percutaneous connector that must be attached to a cable by a trained technician; and the implanted MEA sensor, while able to provide useful signals for years, is generally considered to have variable reliability and stability. Further, the design of this MEA may not provide a sufficient number or form of needed signals over many years. Across this set the interface and decoding problems are paramount.
These ( ) shortcomings are not surprising. Creating BCIs to restore movement or communication is an exceptionally complex task that requires the integration of fundamental neuroscience, computational theory and algorithms, and technology, and must also include clinical needs, regulatory science and health care delivery system policy, and consideration of their ethical, legal, and social implications ( Fig. 25.2 ). After this initial wave of proof-of-concept translation to humans it is now particularly important to assess these accomplishments and identify and act on the key basic neuroscience, computational, and technology advances to transform current lab-based BCIs into available medical devices.
The intention of this chapter is to address key aspects of the question: Why are BCIs not better? I will give a broad outline of the state of the field and then try to distinguish major limits to progress toward a commercially viable and clinically meaningful device. The aim is to provide a stimulus to accelerate the development of BCIs from a proof of concept to a real-world medical device available to a wide range of people with various forms of paralysis (from severe to mild) whose lives could be significantly enhanced through this technology. The chapter will focus on four challenge areas:
- •
goals for BCIs
- •
neuroscience knowledge
- •
computational processing abilities
- •
technology for a clinically useful product
iBCIs, whether to restore locomotion, communication (from typing to speech), or reach-and-grasp arm actions, share many of the same challenges. This chapter will largely deal with challenges for iBCIs that re-create functions achieved by the arm (spanning reaching and grasping of the user’s own arm to operating a computer, including communication as if the user were employing a keyboard and mouse with his or her hands), where most of the research and development effort has been directed. Nearly all of the issues described are readily applied to other body systems; these all have substantial value and could benefit by being considered in the same framework presented here for the arm. It is noteworthy here that significant progress in lower limb control by iBCIs is being made, including technology that could create a brain–spinal interface for body reanimation ( ). My goal is to point out some of what I feel are the most important challenges to tackle now. Other aspects of the very large literature on BCIs are covered in the following reviews: . While useful for simpler applications, by and large external BCIs, which lack access to information-rich neural signals (described under Signals: What to Record? section), cannot provide sufficient information needed to approach natural, complex multidimensional arm and hand control or communication speeds—a potentially achievable goal for iBCIs.
Brain–Computer Interface Clinical Goals
Setting a well-defined goal is a key first step for BCIs to shape and focus critical evaluation of shortcomings as well as guiding progress. In hindsight, those in the field have sometimes debated, for example, performance goals for BCIs with different intended uses. For example, bit rate has been argued across BCIs that have different levels of automated control and disparate long-term visions. BCIs with different intended clinical applications can be valuable for different purposes but require different success metrics. From the user application perspective, BCI goals can range broadly from providing a single-state switch to the full restoration of complex limb movement, as schematized in Fig. 25.3 . To a person with severe paralysis, unable to move at all or to speak, a reliable switch (one trustworthy bit) is powerfully enabling—it can allow communication that is otherwise impossible. Devices, like that based on a P300 speller with a wearable EEG cap, have already reached the commercial stage ( www.intendix.com ). As such, they are an important step to demonstrate the translational possibilities for BCI technology in general. By contrast, for a person limited only in his or her ability to make useful hand grasps a switch is not worthwhile because such actions are already achievable in many other ways. A BCI to restore dexterous grasp, especially if it required surgical implantation in the brain, would, however, need to deliver performance closer to the reliability, speed, and flexibility of natural control to be acceptable. The goal axis presented in Fig. 25.3 can be seen as a way to distinguish different BCI types and applications as well as a tool to identify common scientific challenges and differences. It should be seen as a “value” axis to judge the potential worth of the technology to the user.
Without specifying different goals it is difficult to agree upon acceptable performance criteria or a particular implementation. Setting goals is also critical to establish clinical paths and assess risk. If a goal is to achieve reliable single-state switch control, the success of external BCIs in that realm would argue that a complex device surgically placed in the brain is not warranted. Nevertheless some might elect to receive a surgically implanted device if it were unobtrusive (inside the body) and available all the time, rather than requiring frequent reapplication by a caregiver, as is required for a common EEG cap. However, if the ultimate BCI goal is full restoration of limb movement, and one (reasonably) agrees that the intracranial signals are the only source of such commands, then a more elaborate system is needed. When comparing the value of BCI technologies it is important to consider that a succession of technologies that may be worse in many ways at earlier stages than an alternative, such as current iBCIs, may make sense, and be essential, as a rational stepping-stone toward a more complex goal. Thus, Fig. 25.3 presents a schema to evoke better goal setting. Establishing clear sets of goals, and accepting that there can be a range of goals for different purposes, can help lead to better BCI systems and accelerate progress, leading to better products for BCI users.
Basic Elements of a Brain–Computer Interface
There is general agreement on common system features for BCI systems of all types. BCIs require a neural interface (a “probe,” electrode, or signal detector) for signal acquisition (or to provide feedback in the case of stimulation probes), signal processing hardware that can amplify and condition (e.g., filter, digitize) neural signals, communication devices to transmit signals to computational devices, a decoder (algorithms and associated computational and other hardware) to transform neural activity patterns into a command, and any of a wide range of controlled devices, generally known as assistive technologies (ATs). Various versions of the block diagram for BCI system design of more or less complexity have appeared beginning about 1973 ( ). BCI systems include connections to physical surrogates for the arm, such as a robotic limb or an exoskeleton that supports arm movement, or a computer (for communication); these form the main AT types. A prosthetic hand can also be considered a BCI end effector because it is another physical surrogate controller. In addition, an FES system intended to bridge the BCI device to the arm could also be considered an AT in the service of generating natural arm movements based on BCI commands. The design of a BCI system is unusually multifaceted: it requires integration of knowledge and expertise from neuroscience, computation, and technology ( Fig. 25.1 ). In addition, clinical, commercial, regulatory, and health care considerations as well as ethical, legal, and social implications (ELSI) are critical to the implementation of a device that can eventually be made available and has value to users ( ). Some of the most timely challenges to iBCI progression are considered in the next sections, but of all the challenges as of this writing, the most formidable are the neural interface and the decoder.
Fundamental Neuroscience Advances
Signals: What to Record?
Better knowledge of neural coding could improve BCIs. Roughly speaking, two types of electrical signals can be recorded from the cortex—spikes and field potentials (FPs) (see, for authoritative review, ). It is broadly accepted that single-neuron spiking, or action potentials, represents the fundamental neural communication code (see, e.g., , for an in-depth discussion of these issues); the series of spike events in time captures considerable information in its rate (number of impulses/time bin), although codes related to timing relationships are possible. Spike rate variability can be considered as noise or potentially as input from another source, an important and complex fundamental issue beyond the current discussion. Combining neuron activity together, as ensembles or other collective measures, adds to the information available from the spike rate of a single neuron. The way to combine neurons for optimizing information and the scale of these operations are deeply linked to our understanding of information processing in the nervous system ( ). Under the model that spiking is the fundamental code, spiking-related signals, especially from a sufficiently large population of central nervous system (CNS) neurons, might provide all of the information needed to understand what actions are being performed. That is, one could understand the relationship between an observed spiking pattern and a desired action using the correctly sampled population of neurons. Importantly, the size and composition of a “sufficiently large population” are open to debate, in part because information is distributed across multiple scales and across multiple cell types from neurons to networks. Factors like underlying state fluctuations, activity-dependent plasticity, or context sensitivity also make the task of decoding more complex. This set of problems lies in the domain of neuroscience, where a theory of the neural code is an essential part of formulating a decoding scheme. While most might agree that more spike channels (e.g., larger ensembles) are better, any discussion of spike sampling is inextricably linked to the limits of the possible technology (see Technological Challenges section), which can inlcude only dozens to hundreds of neurons for iBCI devices in human investigational trials as of this writing.
Unlike the time series nature of spiking, FPs are a continuous signal emerging from a complex mix of synaptic inputs, but also often including spiking information in higher FP frequencies ( ). Power spectral analysis long ago demonstrated that FP spectral power reflects more global brain states, such as sleep states or alertness. FP responses can also reflect massed activity related to a sensory or cognitive input, as well as arm action variables. Conclusions about the “meaning” of FP signals are made more complex because their origin and information content depend on the size or configuration of the sensor used (bigger surface area electrodes summing over larger areas) and the way the signal is processed, such as the form and bandpass settings of filters used. FPs have different names based on the site at which they are recorded: EEG if outside the skull, electrocorticogram (ECoG) if recorded on the surface of the brain (or on the dura), local field potentials (LFPs) if recorded intracortically; intracranially recorded FPs are also called “intracranial EEG.” Caution is warranted because this terminology mixes implications about signal source and the type of technology used.
FP signals are complex. LFPs are likely to have local information because they are recorded from a small tipped microelectrode, but large global signals from distant (or local) sites could appear in the LFP, depending on how the signal is processed (e.g., common mode rejection amplifier design, reference electrode placement). Similarly, higher frequency band FP features can depend on filtering and signal processing procedures, contributing to differing views on the information content they contain ( ). Because spikes are highly specific and detailed in information, whereas FPs more a mix of local and more global signals, spikes would seem to be ideal signal source for commands. However, FPs provide movement information ( ), as well as global state information not necessarily apparent in the spikes of individual cells, which means they too could be a useful signal source alone or in combination with spike measures. At least one comparison shows that information about reach and grasp spans FP and spiking signals. Overall, however, data from the motor cortex of able-bodied monkeys support the idea that spiking is the most information-rich signal for reach and grasp ( ). While both FP and ensemble spike signals unquestionably contain movement information, neither of them, given current sampling methods, provides sufficient information to fully reconstruct all of the dexterous actions of the hand that it regularly makes at the speeds at which it ordinarily operates. Research is still active in determining the best possible signal, or signal combination, to code, or “represent,” for these actions. If FPs are only marginally less than spikes in terms of information content and if FP recording is a much simpler technology to use (see later), then BCIs based on that approach might be more practical; on the other hand, better understanding of the nature of spiking could sway the choice in that direction. The nature of the neural code is therefore not only of intrinsic interest to understand how the brain works, it also is very important to shape the future design of effective BCIs.
As stated earlier, the decision of which signals to record is also entwined with technological issues of how to record them. Penetrating electrodes, so far, do not reliably maintain recording of the same cells across days; this shifting population means that a decoding model is unpredictable and unstable across days and the model must be re-created because different cells appear each day. By contrast FPs can be recorded from the cortical surface and may have enough information about a local collection of spiking neurons in their higher frequency signal that they would be a better choice. However, this question remains open because the stability of such FP recording devices, especially for many years in humans, has not been established. Indeed, the technology required is not apparent, especially for ECoG electrodes: is it better to have many large electrodes to sample over a large area? Or, is it better to have smaller surface area electrodes for more detailed information? This depends on the nature of information representation. Are larger electrodes able to provide a more stable signal than smaller ones? Data from monkeys suggest that ECoG FP signals may be sufficient for some levels of control over long times ( ) and that properly designed ECoG electrodes can even capture aspects of spiking ( ). These findings suggest that surface electrodes may be adequate command signal sources to meet some BCI objectives. Thus, progress in this technology and the information the electrodes provide in long-lasting, stable, and reliable interfaces are likely to be essential to much better iBCIs, but these remain unanswered and critical questions. In addition to the formidable challenge of reliable command signal decoding from neural signals, a neural interface that can provide a reliable, stable, high-information-content signal to the decoder is the most significant barrier to progress for BCIs.
Finally, two further concerns about neural signals for applications in people with damage to motor pathways are important to note. First, both FPs and movement-intention-related spiking activity remain in the human cortex years after paralysis onset and, second, changes in neural activity are immediately produced by imaging arm actions, at least superficially not unlike activity seen in the motor cortex of able-bodied monkeys ( ). This means that signals are available as command sources even after injury or during progressive ALS paralysis and that previous and ongoing work in nonhuman primates is highly valuable to improve iBCIs for people.
Location: Where to Record?
A second form of new neuroscience knowledge that could improve progress in developing BCIs is a better understanding of the cerebral motor system. Coordinated limb actions result from the collective interactions of neurons across widespread cortical networks encompassing perhaps a dozen or more frontal and parietal areas ( Fig. 25.4 ) ( ) (as well as a large part of the rest of the nervous system, from brainstem to cerebellum to basal ganglia and thalamus, shamefully ignored here). Some parts of the broader motor network are closer to planning or processing of perceptual or cognitive information, and information about reach and grasp may well be separated into separate divisions of the network ( ). It has been known for well over a century that the primary motor cortex (M1) is a final node in the path to provide motor control signals and that it is divided into leg, arm, and face subdivisions. Electrical stimulation of this area produces movement at the lowest thresholds across cortex; M1 is the source of the majority of connections to the spinal cord, and lesions of this area produce profound paralysis of skilled, volitional, independent movements (particularly of the distal musculature). Lastly, M1 spiking activity in nonhuman primates correlates with and precedes movement. Thus, the M1 has been a logical target for arm movement BCIs. Although the M1 is demonstrably an effective signal source, BCIs based on signals across the motor network provide control signals that would give better control. Andersen and coworkers have shown that signals from the human parietal cortex can be decoded into neural motor commands ( ). However, the information provided by signals from either the M1 or the parietal cortex is not enough to re-create coordinated, flexible reach and grasp at typical movement speeds.
Related posts:
Fundamentals of Burst Stimulation of the Spinal Cord and Brain
Gene-Based Neuromodulation
Spinal Cord Stimulation for the Treatment of Low Back Pain
Epilepsy : Anatomy, Physiology, Pathophysiology, and Disorders
Colonic Electrical Stimulation for Constipation
Transcranial Alternating Current Stimulation and Transcranial Random Noise Stimulation
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
