Motor intentions are divided into different types depending on the different abstract levels. Consider the reaching motion of monkeys, such as a self-feeding motion, as an example. This motion consists of the following movement sequence:

Various types of physical values such as the EMGs of each muscle, grip force, angular velocities of joints, three-dimensional wrist positions, and hand postures can be measured. Based on the interpretation of these physical values, the movement phase (e.g., waiting, reaching, grasping, resting) can be considered as the motor intention of the higher abstract level. Of course, the monkey’s intention in performing the reaching movement is one of the motor intentions of a higher abstract level. In this study, we consider motor intentions of this abstract level as task-oriented motor intentions.

According to the theory of localization of brain functions, information from different abstract levels is processed in different parts of the cerebral cortex. On the other hand, the planning, control, and execution of voluntary motions (VMs) are processed in the motor cortex. In this book, the abstract level of motion intention that will appear as brain activity is discussed based on the brain and the muscle activity measured from a monkey.

A monkey (Macaca fuscata) implanted with EMG and ECoG electrodes is used as the experimental subject. EMG and ECoG signals are recorded simultaneously using a Neural Data Acquisition System OmniPlex (Plexon). The EMG signals are recorded as auxiliary analog inputs of OmniPlex. The band of the signals is limited by a low-pass filter (250-Hz cutoff), and the signals are recorded with a 500-Hz sampling rate.

The target muscles, which are related to the locomotion of the upper limb and hand, for measuring EMGs are listed in Table 12.1.

Figure 12.4 shows the alignment of ECoG electrodes. The target area is around the left motor cortex, including the premotor area (PMA), primary motor cortex (M1), and primary somatosensory cortex (S1).

We designed a lever operation task as a reaching task based on the self-feeding motion of monkeys. Figure 12.5 shows an outline of the task. The monkey is kept under restraint on a chair. At first, a push button (home button) is set up under the monkey’s right hand and a lever is placed in front of the monkey. A tube is introduced into the mouth of the monkey and liquid reward is given through a pump. The pump is actuated when the subject pulls a lever after the home button is pushed. The monkey is adequately trained in performing this task.

The measured EMG signals are transformed to iEMGs (integrated EMGs) as follows:
where S _m (t) is the signal measured by the mth EMG electrode at time step t; M _m (t), the iEMG of the mth channel of the EMG; and L, the term of consideration. Because the iEMG strongly correlates to the exerted muscular force and is robustness to white noise, it serves as an appropriate index of muscular activity.

We already know that some frequency bands are effective for determining the locomotive state of a subject [34]. Table 12.2 shows the range of each frequency band.

Previous studies have shown that high-gamma power strongly correlates to locomotive events in the same way as EEGs or LFPs. However, the range of the high-gamma band differs in each paper. To cover these different definitions, we separate the high-gamma band (80–250 Hz) into two ranges: γ _L (80–150 Hz) and γ _H (150–250 Hz). The power of each band is decided by calculating the power spectrum by short-time Fourier transform (STFT). The window size L equals.

We estimate EMGs from ECoG signals by partial least squares (PLS) regression [35]. Because of the relationship between the spatial resolution of ECoG electrodes and the distances between adjacent electrodes, the signals obtained from the electrodes are collinear. In regression analysis, collinearity makes it difficult to decide the values of regression coefficients and reduces the prediction accuracy. On the other hand, PLS regression serves to remove collinearity and improves the precision of regression analysis.

In this study, PLS regression is performed by the following procedure. First, the feature vectors of ECoGs are constructed as follows:
where x _i (t) (i = 1, 2, …, N) is the sub-vector of the feature vector x _i (t). As defined in Eq. (12.1), each element of x _i (t) indicates the power of the corresponding band. These elements are considered explanatory variables in PLS regression.

where y(t) is iEMG of the target muscle at time step t, x′ _k (t) is the kth element of latent variable vector x′(t) corresponding to x(t), β _k (k = 0, …, r) is the kth regression coefficient, and E(t) is the error term. The vector x′(t) is calculated with the coefficient matrix A to maximize the covariance of y and x′.

We achieved data sequence including 100 of regular trials. The coefficient matrix A is decided by using the data including 90 trials, and prediction accuracy is evaluated with sequential data including ten trials except the data used to decide the coefficient matrix A.

An example of prediction during 2 s is shown in Fig. 12.6. The solid line represents actual values and dashed line represents predicted values. In most cases, trends and peak values are matched well. Though the peak time shift is admitted in some cases, for example, FCU, the rise time is not so largely changed in such cases.

For quantitative evaluation, correlation factor and root-mean-square error (RMSE) for each muscle are shown in Table 12.3. All correlation factors are significant at the 95 % confidence level. Of them, nine factors exceed the value, which can be considered highly correlated (0.7). In the case of TLaH, BLH, and ECR, the correlation values are not so high. However, they make not so much difference comparing RMSE.

As mentioned above, it seems that the iEMG prediction by PLS regression works well. However, some irregular patterns are found during the sequence which has period stability. Figure 12.7 shows the typical regular iEMG pattern and the irregular iEMG pattern during ten trials of continuous reaching motion. In 9 of 10 trials, the regular pattern of actual iEMG is observed. However, a trial which is one of ten trials, irregular pattern is observed shown in Fig. 12.7. On the other hand, the reaching task is performed properly in both of these cases. It has period stability from the task-oriented point of view. Similarly, the brain activity has stability also. The wave form of predicted values is more similar to the regular actual iEMG pattern than irregular one. Though the activity of motor cortex is regular, because of disturbance during reaching motion and so on, the synergy which differs from typical patterns is produced.

We have some other example of irregular iEMG pattern on the reaching task, case of the deltoid muscle of shoulder shows similar irregular iEMG patterns. This is because the target motion is consisted by many types of synergy of muscles. Therefore the intended motion is realized by one of the synergies.

In this section, we would like to show followings: the iEMG identify concrete motion, and it is easy to understand the motion state of subject, but it is too concrete so that it contains error for motion identification. On the contrary, the ECoG signal gives abstract command for motion, so it is good for identification of intentions of motion.

To interpret the phenomenon pointed out above, we describe about contribution of the cerebellum to motor function which is clarified by M. Kawato et al. as follows [36]:

The brain changes these two modes of phenomenon properly and achieves a task. The monkey used as the experimental subject was well trained in the lever operation task. In other words, the monkey performed the motion “programmed” in its cerebellum.

Then, task-oriented motor intentions will be decoded from the cerebral cortex. On the other hand, EMGs will appear as a result of information processing in the central nervous system, which is lower than the cerebellum. In this case, the task-oriented motor intention estimated from the ECoG is superior to that estimated from the EMG.

However, sensory feedback must be carried out with either mode properly. When feedback control is performed, control is not possible without sensory feedback in the first place. Even if feedforward control is performed, the subject will not be able to measure the success or failure of the task. It then becomes necessary to determine the type of relationship that exists among the sensory feedback, brain activities, and measurement of success or failure of the task.

The feeling that an action is real arises from biological feedback from physical interaction with the environment. Neuroprosthetic applications require an interface between the sensors and the muscle system that provides adequate information about the state of the interaction. This information will provide the user with the necessary confidence to move on to the next action. This study aims to develop a biofeedback system that replaces the functions of sensing and action. We have proposed a multichannel ES technique for practical applications of tactile information feedback and reflex muscle drive (Fig. 12.8). Force Sensor Resister (FSR)-type pressure sensors were placed over the fingertips and the palm of the robot hand to detect the moment at which the hand touches an object when performing a grasping task. The sensors in the fingers were connected in parallel to work as a single pressure sensor; the sensors in the palm are placed at three different positions. Eight different sensing positions are used. The stimulator uses a microcontroller with an analog-to-digital converter to acquire sensor signals; whenever a sensor signal exceeds a preset internal threshold, the robot hand is believed to have touch an object, and an ES signal is generated.

This feedback is applied to the prosthetic hand to provide a feeling of grasping. The stimulation electrodes are attached to the remaining body surfaces, e.g., upper left arm or abdomen for a right-hand amputee. The effects of stimulation are revealed through fMRI analysis.

A phantom limb image is a part of the general “body image” of an individual. It also serves as a biological example of brain plasticity by the control of end effectors using sensory feedback. To evaluate the practical use of a prosthetic device, we propose a quantitative index for how the brain adapts to the functions of the device. The effects of adaptation to the prosthetic devices can be observed by changes in the fMRI images. The fMRI analysis reveals interesting phenomena regarding body image plasticity during the training of the EMG prosthetic hand by using sensory biofeedback.

The strength of brain activation is indicated by a stochastic value T under the conditions of certain significance. We focused on the activated patterns at the place in the sensory motor cortex corresponding to the manipulation of the EMG prosthetic hand, and we analyzed the relationship between intensive control and consciousness of tactile feedback. The device usability was evaluated based on the following tactile feedback parameters: place, timing, shape, phantom sensation and mixed sensation, and response and reliability of EMG controller.

To provide visual feedback, we used a video camera to show the prosthetic hand movements inside the fMRI room. The subjects viewed the image through a set of mirrors that reflect the image captured by the video camera. This is necessary because the fMRI cannot be used with the robot hand owing to the strong magnetic forces.

Images were collected using a 1.5-T MRI scanner with a circular polarization coil (Siemens Magnetom Vision Plus, Siemens, Erlangen, Germany). Fifty-four images were obtained per slice over an 8-min period. The images were analyzed by statistical parametric mapping (SPM2) analysis. One subject was recruited for this experiment. Six sessions were performed on different days with a 1-month interval for subsequent tests. Stimuli were displayed on the screen using an LCD projector, introduced into the MRI gantry, and presented to the subject via a mirror mounted on the head coil. One block had six volume scans, and one run consisted of nine blocks that were preceded by six dummy scans. One block consists of three volumes each for the task and the control. The functional images were collected in six runs: one with an echo-planar imaging (EPI) sequence (TR = 3,000 ms, TE = 60 ms, FOV: 192 mm, 64 × 64 image matrix). From each run, 54 sets of 30 slices, 6-mm-thick axial images with 0.6-mm gap between the parallel to the anterior–posterior commissure plane covering the entire brain were acquired. Data analysis was performed using SPM2. fMRI data were realigned and smoothened using a Gaussian kernel applied at intervals of 8 mm, following which statistical analysis was performed using a general linear model with statistical significance defined as corrected P < 0.005.