Overview of This Chapter

Brain–computer interfaces (BCIs) measure brain activity, extract features from that activity, and convert those features into outputs that replace, restore, enhance, supplement, or improve human functions.

BCIs may replace lost functions, such as speaking or moving. They may restore the ability to control the body, such as by stimulating nerves or muscles that move the hand. BCIs have also been used to improve functions, such as training users to improve the remaining function of damaged pathways required to grasp. BCIs can also enhance function, like warning a sleepy driver to wake up. Finally, a BCI might supplement the body’s natural outputs, such as through a third hand.

Different techniques are used to measure brain activity for BCIs. Most BCIs have used electrical signals that are detected using electrodes placed invasively within or on the surface of the cortex, or noninvasively on the surface of the scalp [electroencephalography (EEG)]. Some BCIs have been based on metabolic activity that is measured noninvasively, such as through functional magnetic resonance imaging (fMRI).

This chapter is focused on providing an overview of noninvasive BCIs. After a brief review of the relevant aspects of EEG and fMRI, each of the subsequent sections is dedicated to one of the four different purposes that a BCI may serve and that have been realized as of this writing.

Electroencephalography

EEG sensors detect the coordinated activity of large groups of neurons—the electrical signature of individual or only a few neurons is not detectable by electrodes outside the skull. EEG sensors are usually placed in an electrode cap that is designed to position the electrodes over specific brain regions. Some work has presented EEG electrodes in headbands, headphones, glasses, or other less obtrusive headwear. For many years, EEG electrodes were usually composed of silver/silver chloride rings that were housed in a plastic disk. Electrode gel was needed to establish an electrical connection between the scalp’s surface and each electrode. Work has validated dry electrodes that eliminate the time and inconvenience of gel ( ), but to what extent dry electrodes provide stable EEG, in particular in uncontrolled environments and when used by nonexperts, is still unclear.

Different types of features can be detected in the EEG and may serve as the basis for BCIs. One of the most important of these features is oscillatory activity in different frequency bands: delta (less than 4 Hz), theta (4–8 Hz), alpha (8–12 Hz), beta (18–25 Hz), and gamma (greater than 30 Hz). While the origin of oscillatory activity is still debated, oscillations probably reflect interactions between the cortex and the thalamus or other subcortical structures. Delta activity is most prominent during deep sleep when high-amplitude delta waves can be prevalent over many areas. Theta activity is prevalent during light sleep and meditation. Alpha activity increases over occipital areas when people rest with their eyes closed and during light sleep, and (along with theta and beta) may be used in BCIs to indicate workload or concentration. The phenomenon of “alpha blocking” refers to the decrease in alpha activity that occurs when a person is asked to open the eyes and perform a complex task. Because this is one of the most obvious changes in the EEG that people can easily produce, users are often asked to alternate between eyes-closed relaxation and eyes-open concentration to confirm that their EEG system is working properly. The changes in EEG activity during sleep are driven largely by activity in the pons, thalamus, and occipital regions. Activity in the same alpha frequency range, but detected over sensorimotor instead of visual areas, is called the mu rhythm. The mu rhythm is modulated by expected, actual, observed, or imagined motor movements or associated sensations. These changes in mu activity have been called event-related (de-)synchronization or ERD/S (see Fig. 26.1 ), and have been widely used in BCIs.

(A and B) The changes in mu activity centered around 12 Hz for (A) actual and (B) imagined right-hand movements. The colors reflect the proportion of the signal variance accounted for by the task. These two images show that imagined movements produce changes that are less pronounced than those resulting from actual movements, but show a similar topographical distribution. (C) EEG power over site C3 for a different subject who rested ( dashed line ) or performed right-hand movement ( solid line ). The mu activity at about 12 Hz and its harmonic around 24 Hz are both greatly reduced by movement. (D) The resulting r ² correlations for rest versus movement. This image also shows that movement primarily affects power in the mu frequency bands and its harmonics.

From Schalk, G., McFarland, D.J., Hinterberger, T., Birbaumer, N., Wolpaw, J.R., 2004. BCI2000: a general-purpose brain–computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51 (6), 1034–1043.

Beta and gamma activity is most apparent during concentration and can also include harmonics of mu activity ( ). These frequency bands have been used in BCIs to detect concentration or information overload. Both bands are often divided into high and low, and low and high bands can reflect more details of the brain dynamics underlying cognition and emotion. While the source and purpose of the brain’s different oscillatory activities are not fully understood, they seem to generally reflect thalamocortical interactions (primarily through layers 4 and 5 of the cortex) to coordinate activity across different regions and neural populations ( ).

In addition to oscillatory activity that is detected in the frequency domain, electrophysiological activity in the time domain also reveals useful information. When activity is time locked to a stimulus, activity changes following the stimulus are called event-related potentials (ERPs). Because ERPs that result from only a single stimulus are usually too noisy to be detected, both researchers and BCI systems typically repeat the task and associated stimulus several times to acquire several ERPs that can be averaged together, resulting in a clearer signal. ERPs are often named according to their electrical valence (positive or negative) and time in milliseconds from the relevant event. For example, the P300 ERP reflects a positive change in voltage of about 300 ms after an event, and reflects cognitive processing of that event. The P300 has different subcomponents, notably the P3a and P3b, that each reflect different aspects of task processing. The frontally prominent P3a is largest when processing novel stimuli, and reflects attentional alerting and the need to update working memory. The more parietal P3b reflects memory updating and planning a response, such as pressing a button or counting. Concordantly, the sizes of a person’s frontal and parietal areas are correlated with the amplitude of the P3a and P3b, respectively. The P300 reflects contributions from other cortical and subcortical regions as well, including the hippocampus, anterior cingulate, and medial temporal lobes. Earlier components, such as the P100 and N170, instead convey early perceptual processing, and show activity in earlier processing areas such as V1 (primary visual cortex) ( ).

Time domain activity may also be detected prior to an anticipated event, such as a button press. Before a voluntary movement, the readiness potential (RP; also called Bereitschaftspotential or BP in German) will change across two stages (see Fig. 26.2 ). About 1.5 s prior to the movement, the supplementary motor area (SMA) and related motor preparation areas exhibit a slow bilateral negative change in voltage. About half a second prior to movement, a much sharper change is apparent contralateral to the movement in the SMA and the primary motor cortex (M1). These two stages seem to reflect movement planning and execution, respectively. The RP is one type of movement-related cortical potential (MRCP), a family of signals that can index movement speed, force, effort, precision, training, complexity, concentration, and other factors ( ).

Different components of the readiness potential, also called the Bereitschaftspotential (BP), are shown. The rightmost vertical line reflects the onset of a voluntary movement. In this image, the voluntary movement was self-paced tapping of the right index finger. The BP phase shows a slowly developing negativity from about 1.5 to 0.5 s prior to the voluntary movement, which becomes more pronounced during the period 0.5 s prior to the movement ( ). MRCP , movement-related cortical potential; NS , negative slope.

Another time domain EEG phenomenon is the contingent negative variation (CNV). The CNV is a bilateral negative change that is prominent over the top of the scalp, and primarily reflects activity from frontal areas. The CNV reflects slow changes, on the order of a few seconds, that can occur between a warning stimulus (which informs someone that a relevant stimulus will soon be shown) and an imperative stimulus (reflecting that someone needs to take action). The CNV was discovered over 50 years ago ( ) and has been extensively studied. It can reflect a variety of factors, including emotional changes, focused attention, general arousal, and the stimuli’s expectancy and perceived relevance, probability, intensity, and timing. However, it has not been widely used in BCIs because other types of signals described here are generally more reliable, require less training, and allow higher bandwidth communication.

Rapid presentation of visual stimuli (such as flickering LEDs or objects on a computer screen) can result in steady-state visual evoked potentials (SSVEPs). SSVEPs reflect the rapid firing of visual cortical areas, primarily V1. If the user focuses attention on one stimulus, EEG signals over visual areas increase in power at that frequency and its harmonics. This allows BCIs to detect which stimulus the user chose to attend to (see Fig. 26.3 ).

Steady-state visual evoked potential (SSVEP) activity elicited during selective attention to two oscillating checkerboards, each of which oscillated at 6 or 15 Hz. (A and C) Spectral power for one subject over site O1 (A) or O2 (C). The solid and dotted lines show activity elicited while the subject focused on the 15- or 6-Hz checkerboard, respectively. (B and D) The r ² values that reflect the correlation between different frequencies and the instruction to focus on either target stimulus. (E) A topographic map of these differences. It is shown that selective attention to a flickering stimulus increases power at the eliciting frequency and, to a lesser extent, the harmonics of that frequency. The SSVEP activity is much more pronounced over occipital areas than over other sites.

From Allison, B.Z., McFarland, D.J., Schalk, G., Zheng, S.D., Jackson, M.M., Wolpaw, J.R., 2008. Towards an independent brain–computer interface using steady state visual evoked potentials. Clin. Neurophysiol. 119 (2), 399–408.

If different stimuli are presented at the same frequency but different phases, a BCI may also infer the attended stimulus based on phase measurements in the EEG ( ) or their autocorrelation with an m-sequence in a variant of SSVEPs called code-based VEPs or c-VEPs ( ).

Vibrotactile stimuli can elicit steady-state somatosensory evoked potentials (SSSEPs), and thereby may provide the basis for BCIs for persons without vision ( ). Steady-state auditory evoked potentials (SSAEPs) have also been studied. Consistent with other somatosensory evoked potentials (SEPs), SSSEPs and SSAEPs involve activity in the corresponding primary cortical sensory area in tandem with higher sensory areas and relevant thalamic nuclei (lateral geniculate, visual; medial geniculate, auditory; ventral posterolateral, somatosensory signals from the body). SEPs have many clinical and research applications, primarily exploring lower-level sensory processes. SEP research has also been used to study schizophrenia, depression, attentional deficits, epilepsy, and other conditions ( ).

Metabolic Activity

Techniques that measure metabolic activity detect changes in blood oxygenation or other indirect measurements of neuronal activity. Unlike electrical changes that immediately reflect the activity of neuronal populations, metabolic changes typically occur a few seconds after neuronal activity changes. Despite this inherent lag, some BCIs have used metabolic changes in successful demonstrations.

The two most common imaging techniques that can detect metabolic activity are fMRI and positron emission tomography (PET). FMRI and PET are volumetric imaging techniques, i.e., they can detect changes deep in the brain that are invisible to most electrical methods (see Fig. 26.4 ). At the same time, they require expensive and heavy equipment and they each incur other practical challenges: fMRI requires a very powerful magnetic field that is unsafe for some patients, and PET requires the injection of radioactive tracers. Functional near-infrared spectroscopy (fNIRS) also detects changes in blood flow and does not have these disadvantages. It is safe, portable, and relatively inexpensive, although, like EEG, it is limited to the detection of activity near the brain’s surface. FNIRS requires placing a device on the surface of the scalp that includes an emitter and several detectors. The emitter shines light through the scalp; this light is reflected off of the cortex, and reflection parameters are changed depending on local cortical activity.

These fMRI images show how a person with attention deficit hyperactivity disorder (ADHD) exhibits different activity compared to a healthy control. Moreover, they show how fMRI can reveal correlates of brain function well below the surface of the cortex, which are difficult or impossible to detect with most other methods. However, as of this writing, fMRI systems are practical only in hospital settings.

From ucdmc.ucdavis.edu (2014).

Introduction

BCIs for replacing lost functions have been explored primarily to help persons with conditions that impair most or all voluntary movements, including persons with late-stage amyotrophic lateral sclerosis (ALS) or tetraplegia. For individuals struck by these conditions, BCIs may replace lost functions (such as communication or movement control) by using brain activity to control an artificial effector (such as a robotic arm or a communication system). The following sections give an overview of BCIs for communication or control that have been developed as of this writing.

Communication Functions

Control Functions

Future Directions

Several promising directions for noninvasive BCIs could replace lost functions in different patient groups. For persons with DOC, noninvasive BCIs could go beyond basic assessment and communication to provide more detailed assessment of cognitive function, rehabilitation, outcome prediction, more complex communication such as spelling, and basic environmental control ( ). For example, detection of SSSEPs associated with tactile stimuli could allow several choices for patients who cannot see ( ). Patients may benefit from BCI systems for bladder or bowel control ( ).

Introduction

BCIs may be used to restore a patient’s ability to control his or her body. This category of BCIs aims solely to help persons with disabilities. Unlike BCIs that replace function, which control external devices (such as a robotic arm), BCIs that restore function eventually move the body’s own limbs. The goal is to bypass damaged pathways that connect to functioning effectors, such as the patient’s arms and hands. This technology could benefit people with stroke, brain injury, spinal cord injury, and other conditions that damage the brain or spinal cord.

A substantial volume of work has focused on restoring function to the arms and hands. This is a prevalent need for many patients and is a relatively safe research direction (restoring function to lower limbs adds the risk of falling), and stimulating nerves or muscles in the arm to initiate hand grasping may be simpler than producing the extremely intricate and coordinated muscle activations necessary for walking or speaking. The following section gives an overview of devices that have been used to produce limb movements.

Devices That Produce Limb Movements

Brain–Computer Interfaces for Restoration

Upper Limb

The Freehand neuroprosthetic is an implanted FES device. In relatively early work, it was combined with a 64-channel EEG-based BCI to restore grasp function ( ). Later work allowed a single patient to use the Graz motor imagery approach, which had already been validated for orthosis control ( ), with the Freehand system ( ). Fig. 26.5 presents examples of two patients using different types of FES systems with BCIs to restore grasp control.

(A) A patient using an electrode cap to detect movement imagery that causes a noninvasive (transcutaneous) functional electrical stimulator (FES) system to activate and trigger grasp function. (B) A different patient who instead uses an invasively implanted FES to control grasp.

From Muller-Putz, G.R., Scherer, R., Pfurtscheller, G., Rupp, R., 2006. Brain–computer interfaces for control of neuroprostheses: from synchronous to asynchronous mode of operation. Biomed. Tech. (Berl.) 51 (2), 57–63.

Many groups have explored other issues relating to functional electrical stimulation and upper limb movement. Other early work ( ) compared the efficacy of EEG- to EMG-based control of an orthosis. In one study exploring long-term use, a single C4 spinal cord injury (SCI) patient used EEG-based motor imagery measures to control both an FES and an orthosis to restore hand function. After a year of training, his motor imagery accuracy remained at only 70%, but he still found the device useful ( ). For patients who cannot attain good accuracy with motor imagery, SSVEP activity was validated for orthosis control. Seven healthy volunteers without prior training were told to focus on one of two flickering LEDs placed on an orthosis to either open or close it (see Fig. 26.6 ). Six volunteers attained good accuracy in an asynchronous control paradigm, although the system produced a high rate of false positives, and some participants did not like the flickering lights ( ). Related work addressed these problems with a hybrid motor imagery–SSVEP system for orthosis control. This BCI system allowed subjects to use motor imagery as a “brain switch” to switch the LEDs on or off, thus enabling users to eliminate both problems. Four of six healthy participants attained good accuracy ( ). found that proprioceptive feedback led to improved motor imagery control of a robotic exoskeleton (compared to no feedback).

The left (A, top) shows an orthosis that can be opened or closed by moving along the axes shown by white arrows via steady-state visual evoked potential activity elicited by either of the black LEDs (B, left middle). The remaining three images show the orthosis moving throughout three stages of opening and closing. The right shows a healthy volunteer using the orthosis to move a bottle with a white foam shield.

From Ortner, R., Allison, B.Z., Korisek, G., Gaggl, H., Pfurtscheller, G., 2011. An SSVEP BCI to control a hand orthosis for persons with tetraplegia. IEEE Trans. Neural Syst. Rehabil. Eng. 19 (1), 1–5.

Lower Limb

BCIs to restore control of the lower limbs often use EEG-based imagination of walking, as this provides a natural and intuitive mapping between thought and action and is relatively easy to detect with trained users. One patient with paraplegia resulting from SCI learned to either relax or imagine walking while EEG measures of these activities were used to drive a lower-limb orthosis ( ). One study with healthy volunteers validated BCIs to start or stop an exoskeleton by thinking about gait initiation or termination ( ). In a proof-of-concept study, three healthy people used EEG-based measures of gait initiation to direct an exoskeleton ( ) (see Fig. 26.7 ). classified different types of lower-limb movements (hip, knee, and ankle) as people trained to control a virtual avatar’s gait over 8 days. The users’ attention during gait was also explored using different EEG frequency bands, which could influence feedback during system operation ( ).

An EEG-based system to control a lower-limb exoskeleton being used by a healthy volunteer (left) and a spinal cord injury patient (right). The work with the patient requires safety rails and nearby staff because of the risk of falling.

From Lopez-Larraz, E., Trincado-Alonso, F., Rajasekaran, V., Perez-Nombela, S., Del-Ama, A.J., Aranda, J., Minguez, J., Gil-Agudo, A., Montesano, L., August 3, 2016. Control of an ambulatory exoskeleton with a brain–machine interface for spinal cord injury gait rehabilitation. Front. Neurosci. 10, 359. http://dx.doi.org.easyaccess1.lib.cuhk.edu.hk/10.3389/fnins.2016.00359 . eCollection 2016.

Introduction

The BCIs described so far focus on people with disabilities who seek to replace or restore lost functions. BCIs may eventually also benefit healthy persons by enhancing the capabilities of the central nervous system to enable people to perform tasks better, faster, or more easily. BCIs to enhance function are generally “passive,” meaning that the user does not actively have to perform mental activities devoted to controlling the BCI (such as imagining movements or counting flashes or tones). Instead, passive systems measure activity while the user is performing other tasks. Thus, BCIs to enhance function generally aim to provide additional capabilities without disturbing or distracting the user.

User State

A substantial body of work since 2007 has shown that the EEG offers indicators of arousal, engagement, workload, emotional valence, and other factors. Accurate real-time detection of these indicators could be used to modify a user’s ongoing interaction with a computer or another external device. This direction has been explored for decades, but is still largely confined to the laboratory.

detected alpha and theta changes over electrode sites Cz and Pz/Oz while sonar operators performed a simulated sonar detection task, and could predict periods of poor task performance. demonstrated a wireless four-channel EEG system that could detect alertness lapses and sound a warning. In a simulated driving environment with 11 healthy participants, discriminated effective from ineffective warning tones using EEG spectral activity. The authors later extended this approach ( ), which might lead to other mitigation measures for drivers or others who do not notice warning stimuli. In these examples, the BCI enhances natural function by reducing the risk of disaster resulting from alertness lapses.

BCIs could also enhance users’ ongoing interaction with software. This approach could reduce errors or keep users engaged, such as by increasing difficulty if users are bored ( ). For example, research from NASA explored real-time “adaptive automation” using EEG-based measures while people performed tasks from a multiattribute test battery at different difficulty levels, with the goal of adapting the user’s workload to reduce overload ( ). The Defense Advanced Research Projects Agency (DARPA)-funded Augmented Cognition Program also aimed to learn more about system operators’ mental states based on EEG and other measures to adapt the system’s interaction with that user ( ). Automatic adaptation to the state of a user has been further explored in aviation ( ). presented a system that could adapt ongoing interaction with a P300 BCI system based on EEG measures of task load.

BCIs have also been used to enhance games and creative applications. In , healthy people played World of Warcraft through conventional means (keyboard and mouse). The system changed the player’s game character between an elf and a bear based on EEG-based evaluation of stress (see Fig. 26.8 ). and described EEG-based systems to detect bluffing in a game environment and perceived loss of control. described a BCI that can produce music in real time based on EEG measures of user emotion.

The EEG-enhanced World of Warcraft system from . The left shows how EEG-based measures of stress can change the elf character into a bear. The right shows a person playing the game.

Error Detection

People inevitably make mistakes while using BCIs or other technologies. Errors may go undetected or uncorrected, or users need to perform some corrective action that requires time and attention. EEG measures can reveal error potentials, such as event-related negativity (ERN), if people believe they just made a mistake. Real-time detection of ERN or similar EEG features could be used to correct errors or other goals ( ). recorded activity from 64 EEG channels while four subjects used mu and beta activity to direct a cursor to the word “yes” or “no,” and established the difference in EEG activity recorded during the trials in which the subjects did or did not succeed in controlling the cursor to the correct target. One hundred eighty milliseconds after subjects received feedback indicating an erroneous selection, they exhibited a strong positive peak, followed by a negative peak, which was most prominent over Cz ( Fig. 26.9 , left). The authors estimated that, if error activity were used for error correction within their yes/no BCI system, the system’s information transfer rate would improve by 0% to 21% across the four subjects. A different group also explored error activity recorded from 64 EEG channels and implications for online error correction ( ). Seven healthy subjects performed a button-press task in a visual discrimination paradigm. ERPs also showed prominent frontocentral activity from 100 to 200 ms after erroneous responses only ( Fig. 26.9 , right), which could reduce errors by −6% to 49% across the seven subjects.

The left and middle show error-related activity from . The left image shows event-related potentials over site Cz, and the middle image shows the scalp projections for 40 ms around the positive peak for each subject. The right shows error activity from from 100 to 200 ms after feedback presentation. These images indicate prominent central activity 100–200 ms after feedback presentation.

Only gold members can continue reading. Log In or Register to continue

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)

Related

Related posts:

Fundamentals of Burst Stimulation of the Spinal Cord and Brain Gene-Based Neuromodulation Spinal Cord Stimulation for the Treatment of Low Back Pain Epilepsy : Anatomy, Physiology, Pathophysiology, and Disorders Colonic Electrical Stimulation for Constipation Transcranial Alternating Current Stimulation and Transcranial Random Noise Stimulation

Stay updated, free articles. Join our Telegram channel

Join