The first BMI research was reported by Farwell and Donchin [2] about 25 years ago. They developed a communication device that displays intended characters inferred by P300 potentials, recorded by scalp EEG and evoked by randomly flashed character. This is presently well known as a P300 speller. The device is able to infer 2–3 alphabetic characters per minute. Following the P300 speller, BMI devices using mu rhythms were reported. The mu rhythm is a rhythmic brain wave in the 8–12 Hz band range observed over the central sites and related to voluntary movement. In 1991, Wolpaw et al. [3] reported on a BMI device with a cursor that moves up and down corresponding to increases and decreases in mu rhythm power. This device usually requires a certain amount of training for users to voluntarily control the increase and decrease in mu rhythm power. Following these two seminal development, many BMI studies were reported using P300 and mu rhythms.

The initial success of EEG-based BMIs easily leads us to expect that ECoGs will provide even better BMI performance. Brain surface (i.e., subdural) electrodes have been clinically used in the field of neurosurgery for more than 30 years to identify epileptic foci of intractable epilepsy [4]. ECoGs have a higher signal-to-noise ratio and detect higher frequency band activities compared to scalp EEGs. The first ECoG study, as a basis for ECoG-based BMIs, was reported by Huggins et al. [5]; Levine et al. [6]. The first quantitative assessment of an ECoG-based BMI reported that one-dimensional cursor control was achieved at a success rate of 74–100 % [7]. Since the one-dimensional decoding was reported, there has been an increase in ECoG-based BMI studies published. Several groups reported two-dimensional decoding [8, 9], which led to two-dimensional cursor control [10]. The brain surface electrodes used to record are also less invasive to brain tissue and superior in long-term recording stability. Chao et al. implanted ECoG electrodes in monkeys for a year and were able to accurately decode the three-dimensional position of the upper arm for as long as one year [1]. They were also able to decode the arm position for 6 months without additional decoder learning. These results indicate the long-term recording stability of ECoGs, which is one of the most important factors for clinical application.

In our investigation of ECoG-based BMIs, we demonstrated that ECoGs within the central sulcus are useful for decoding upper limb movements [11]. We succeeded in real-time control of a robot hand using high frequency band activities [12] and further showed that, even in patients with severe motor disturbance, movement type was decoded using high frequency band activities during imaginary movements. We describe this study in details later.

Cerebral cortical potentials change due to movements and are thus referred to as movement-related cortical potentials (MRCP) (Fig. 5.1). MRCPs occur before the onset of the movements. They consist of four main components. The readiness potential (RP) is a slow negative potential staring 1,500–1,800 ms before movements. The negative slope potential (NS) is a prominent negative potential that is recorded predominantly in the contralateral electrodes 400–500 ms before movement onset. The NS is followed by a premotor positivity and motor potential (MP).

Basic rhythmic waves, such as α waves at 8–10 Hz over the parietal area, during resting states are called cerebral oscillations. Brain activation induces local oscillatory changes in waves of specific frequency bands. An increase in the oscillatory changes is an event-related synchronization (ERS) and a decrease is called an event-related desynchronization (ERD) [13].For example, hand grasping induces ERD mainly in the hand motor area bilaterally with contralateral predominancy in the α-(8–13 Hz) and β-(13–25 Hz) bands from 500 to 1,000 ms before the movement onset. The ERD peaks after the movement onset. In the high γ-band (more than 50 Hz), ERS occurs a few hundred milliseconds before movement onset and peaks after movement onset. These oscillatory changes are called movement-related cerebral oscillatory changes. The cerebral oscillatory changes are observed not only during movements [14] but also during language activities, somatosensory processing, and mental concentration [15–17].

Both movement-related oscillatory changes and MRCPs reflect macroscopic electrical cortical activities related to movement and have close relationships with one another. Appropriate features selected based on their neurophysiological spatiotemporal properties related to movements and their use as input information for neural decoding make it possible to infer movement intension and type.

In the process of providing neurosurgical treatments for certain groups of patients, we sometimes implant grid electrodes for about two weeks directly on the brain surface to identify optimal sites for motor cortex stimulation for intractable pain or to identify epileptic foci for intractable epilepsy. We also sometimes place strip electrodes within the central sulcus to obtain more efficient pain relief during motor cortex stimulation for intractable pain [18]. With the approval of an institutional ethical committee, we have been investigating BMIs using human ECoGs recorded from electrodes implanted in more than 30 patients.

Neural decoding is a key technology of BMIs and there are many decoding methods. We mainly use a support vector machine (SVM), which is a learning machine algorithm for classification that obtains high classification accuracy by adjusting weight parameters to maximize the margin between the groups to be classified [19] (Fig. 5.2).

We measured ECoGs during two or three types of simple motor tasks of the hand or arm, such as grasping, pinching, and elbow flexion and predicted the type of movement based on analysis of single-trial ECoGs using an SVM. We were able to predict movement types on a single-trial basis with an accuracy rate of 70–90 %. Specifically, we first demonstrated that ECoGs from the anterior wall of the central sulcus are useful for the accurate and early decoding of the movement types [11]. Most of the primary motor cortex, which is responsible for the final output portion of motor commands, lies within the anterior wall of the central sulcus. Especially in humans, the anterior wall of the central sulcus has many neurons directly projecting to the spinal anterior horn cells. Such neurons are thought to be related to fine movement control [20]. We suppose that appropriate neurophysiological feature extraction from the central sulcus contributed to our accurate movement decoding.

We also investigated which frequency band of the oscillatory changes most accurately decoded the movement types and showed that normalized power in the high γ-band (80–150 Hz) gave the highest decoding accuracy of all frequency bands from 3 to 150 Hz [12]. We found, even in the case of severely paralyzed patients, that even just imagery of hand movements induces clear high γ-band responses similar to those of real movements (Fig. 5.3).

We further found a difference in the spatial distribution of the high γ-band activity between the patients who can clearly differentiate the imageries of upper limb movements and those who cannot. In the patients who can clearly differentiate the imageries, clear difference was observed in spatial distribution of the high γ-band activities between hand grasping and elbow flexion, but in the patients who cannot differentiate the imageries clearly, there was no significant difference [21]. This finding suggests that the ability to differentiate motor imageries reflects differentiated cerebral neural representation of each movement, providing a new insight into functional reorganization.

Most recently, we found new evidence regarding the mechanism of motor control: cross-frequency phase-amplitude coupling. Before movement onset, the power amplitude of the high γ-band activities couple with the phase of the α-band activities, but immediately before movement onset, the coupling is attenuated [22]. The cross-frequency phase-amplitude coupling might play an important role in control of movement initiation and postures.

We applied this decoding method to an ECoG-based BMI system for real-time control of a robot arm (Fig. 5.4). ECoGs were measured using a 128-channel digital EEG system (EEG 2000; Nihon Koden Corporation, Tokyo, Japan) and digitized at a sampling rate of 1,000 Hz. We introduced successive decoding every 200 ms and a hierarchical decoding and control method. First, Gaussian process regression was used to estimate the likelihood of the decoding feature for movements. When the likelihood was above the threshold level, an SVM was used to infer the type of hand and arm movements. We introduced “transitional states” into robot posture and gradually controlled the robotic arm from the initial posture to the target posture. These hierarchical decoding and control contributed to smooth motion of the robot arm, even if the decoding accuracy was 70–90 %. We also introduced “modular decoding” by using different decoders for hand and elbow movements, which made it possible to simultaneously and independently decode and control the hand and the elbow. The robotic arm was an experimental anthropomorphic hand [23]. The general movement mechanisms and degrees of freedom of the hand mimicked those of a human hand. The hand was equipped with 8 DC motors to independently actuate eight individual tendons in the robotic hand. The eight tendons work in a coordinated manner to accomplish flexion or extension of each individual finger. As a result, we succeeded in the practical voluntary control of the grasping and releasing of objects [21]. Using a successive decoding and control algorithm, smooth robot hand movement was achieved even though the decoding accuracy on a single-trial basis was approximately 70 %. We also showed that the robotic arm was successfully controlled four days after the initial decoder training even without additional decoder training. Although electrode implantation was limited to only a few weeks in this clinical situation, this result supports the notion that ECoG-based BMIs have long-term stability.

In addition to classification-based decoding for the hand posture, we also developed regression-based decoding for the reaching motion of the arm using a Bayesian algorithm known as sparse linear regression (SLR) [24]. If we combine both decoding methods, the SVM for the hand posture and SLR for the reaching motion of the arm, we are able to decode the hand and arm movement simultaneously.

A noninvasive evaluation of an individual BMI performance is indispensable for determining the surgical indication of the invasive BMI treatment. MEG is a potentially noninvasive method for evaluating individual BMI performance due to its high spatiotemporal resolution and neurophysiological compatibility with the ECoG. We investigated the neural decoding performance of three types of unilateral hand and arm movements on a single-trial basis using a MEG [25, 26]. We used an SVM to decode the movement types. The peak amplitudes of the first components of the movement-related cortical fields (pMRCF) after the movement onset were used as decoding features. The neural decoding accuracies largely exceeded chance level in all of the nine healthy subjects that were evaluated. Furthermore, the p and decoding accuracies were significantly correlated (rs = 0.900, p = 0.002) (Fig. 5.5). These results suggested that the neurophysiological profiles might serve as a predictor of individual BMI performance and assist in the improvement of general BMI performance.