In order to capture efficiently human kinematics – using appropriate motion capture systems – the kinematic models of the human arm and the human hand must be described. The kinematic model of the human arm, that we use in this study, consists of three rotational degrees of freedom (DoFs) to model shoulder joint, one rotational DoF for elbow joint, one rotational DoF for pronation-supination and two rotational DoFs for wrist flexion/extension and abduction/adduction. The kinematic model of the human hand consists of 20 rotational DoFs, 4 for each one of the 5 fingers. Regarding fingers we used for the four kinematically identical fingers (index, middle, ring and pinky) three rotational DoFs to model flexion-extension of the different joints and one rotational DoF for abduction-adduction. Human thumb is modeled, using two rotational DoFs for flexion-extension, one rotational DoF for abduction-adduction and one rotational DoF to describe palm’s mobility that allows thumb to oppose to other fingers. The kinematic models of the human arm and hand are presented in Fig. 1.3.

In order to capture the human arm hand system motion in 3D space, extracting the corresponding joint angles (27 modeled DoFs), we used a dataglove for the human hand and a magnetic position tracking system for the human arm. The Isotrak II^® (Polhemus Inc.) magnetic motion capture system used, is equipped with two position tracking sensors and a reference system. The two sensors of Isotrak II, were placed on the elbow and the wrist respectively, while the reference system was placed on the user’s shoulder. Having captured the positions of the human shoulder, elbow and wrist, the inverse kinematics of the human arm can be computed, following the directions provided in [39]. Alternatively for human robot interaction applications, a human to robot motion mapping procedure like the one proposed in [40], can be used. Regarding the human hand, the Cyberglove II^® (Cyberglove Systems), is used to measure the 2 DoFs of the wrist (flexion-extension and abduction-adduction) and the 20 DoFs of the human fingers. The experimental setup that was used to track human arm hand system kinematics, is depicted in Fig. 1.4 and the different motion capture systems are depicted in Fig. 1.5.

In total, we recorded the myoelectric activity of 16 muscles, of the upper arm (8 muscles) and the forearm (8 flexor and extensor muscles). More specifically the chosen muscles are: flexor pollicis longus, flexor digitorum superficialis, flexor carpi ulnaris, flexor carpi radialis, extensor pollicis longus, extensor indicis, extensor carpi ulnaris, extensor carpi radialis, deltoid anterior, deltoid posterior, deltoid middle, trapezius, teres major, brachioradialis, biceps brachii and triceps brachii. The selection of the muscles and the placement of the surface electromyography electrodes, was based on the related literature [16, 41]. In order to achieve easy, portable and fast to use training schemes several researchers have chosen to place the EMG electrodes, in specific regions but in random (not precise) positions [3]. We believe that the next generation of epidermal electronics [42] will make the electrode positioning faster and easier, thus we choose to take advantage of the higher signal to noise ratio, that accurate electrode positioning offers.

EMG signals were acquired and conditioned using an EMG system (Bagnoli-16^®, Delsys Inc.), equipped with single differential surface EMG electrodes (DE-2.1^®, Delsys Inc.). A signal acquisition board (NI-DAQ 6036E^®, National Instruments), was used for signal digitization and data acquisition.

Regarding data processing, EMG signals were band-pass filtered (20–450 Hz), sampled at 1 kHz, full-wave rectified and low-pass filtered (Butterworth, fourth order, 8 Hz), while for the position measurements, which were provided by the position tracking system at the frequency of 30 Hz, an antialiasing finite-impulse-response filter (low pass, order: 24, cutoff frequency: 100 Hz), was used to resample them at a frequency of 1 kHz (same as the sampling frequency of the EMG signals).

After data collection, all EMG recordings, were pre-processed and epochs of data were created. Those epochs included the different reach-to-grasp movements captured during the experiments. Then, all data were resampled at 100 Hz, where each sample at the new frequency (100 Hz) was calculated as the mean value of ten (10) samples of the original frequency (1 kHz). Based on the profiles of the rectified EMG signals at the new frequency, the onset of muscular activations was defined comparing the amplitude of each muscle’s myoelectric activation to it’s relaxed state. Finally, epochs including only muscular activations captured during the actual tasks were created, and were used to formulate synergistic profiles, using a novel statistical representation technique, that we introduced and which we call “Boxplot Zones”.

A boxplot (alt. box-and-whisker plot) is a method to graphically depict groups of numerical data, through the following five-number summaries: smallest observation (sample minimum), lower quartile (Q1), median (Q2), upper quartile (Q3), and largest observation (sample maximum). Boxplot zones were first defined in [34] to visualize muscular co-activation patterns and are an equivalent of boxplots, while more visually informative representation, suitable for the representation of synergistic profiles. Boxplot zones consist of three different layers. The first layer includes the median line, connecting the medians of all boxplots. The second layer includes the box zone (blue zone), connecting the boxes that contain all the values between the lower and the upper quartile, while the third layer includes the whisker zone (white zone), connecting the whiskers that mark the largest and the smallest observation. A direct comparison of a boxplot and a boxplot zone visualization, can be found in Fig. 1.6.

In Fig. 1.7 we present a “boxplot zones” based visualization of muscular co-activation patterns of sixteen (16) muscles (of the upper arm and the forearm), for one subject (Subject 1) executing reach to grasp movements, towards five (5) different positions in 3D space, to grasp three (3) different objects. The muscular co-activation patterns presented in Fig. 1.7 in terms of synergistic profiles formulated with boxplot zones, depict a significant differentiation between the different reach-to-grasp movements, although the same joints of the arm hand system (human upper arm joints and human hand finger joints) are involved, but for a different task. More precisely, if we examine the synergistic profiles (muscular co-activation patterns) across different subspaces (different positions), we notice that the activity of the muscles of the upper-arm (EMG channels 1–8) reflects most of the differentiation. In contrast, if we examine the muscular co-activation patterns across different objects, placed in the same subspace (a specific position), the activity of the muscles of the forearm (EMG channels 9–16) reflects most of the differentiation.

In Fig. 1.8 we present a “boxplot-zones” based visualization of muscular co-activation patterns differentiation, for 16 muscles of the human upper-arm and forearm, for 3 different subjects performing different reach to grasp movements, towards five (5) different positions in 3D space, to grasp a specific object (rectangular-shaped object).

As we have already noted there is a significant differentiation between muscular co-activation patterns associated with different reach to grasp movements. Statistical significance of muscular co-activation patterns differentiation, can be assessed using appropriate statistical tests. More precisely the Lilliefors test (adaptation of the Kolmogorov-Smirnov test) was used to test the null hypothesis that the EMG data – containing the myoelectric activations – come from a normal distribution. The test rejects the null hypothesis at the 5 % significance level (p = 0. 05), so the data are not normally distributed. Thus, we use non parametric tests such as, the Kruskal-Wallis and the Wilcoxon rank sum test, in order to assess the significance of muscular co-activation patterns differentiation, for different strategies.

The Kruskal-Wallis compares the medians of the myoelectric activity of the selected muscles, for different muscular co-activation patterns, and returns the p value for the null hypothesis that all samples are drawn, from the same population (or from different populations with the same distribution). The Wilcoxon rank sum test, performs a two-sided rank sum test of the null hypothesis that data of myoelectric activations with different muscular co-activation patterns, are independent samples from identical continuous distributions, with equal medians.

More details regarding the statistical procedures used, the reader can find in [43]. All tests were performed to check the differentiation of muscular co-activation patterns for the following three cases:

For all sets, confidence levels were set at 95 %. All tests null hypotheses for all three cases were rejected, proving that muscular co-activation patterns differentiate, between different subjects and between different tasks. In Fig. 1.9, we present the means and the confidence intervals of EMG activity across eight muscles of the upper arm and eight muscles of the forearm, for a subject performing reach to grasp movements, towards three (3) different objects. In Fig. 1.10, we present the means and the confidence intervals of EMG activity across eight muscles of the upper arm and eight muscles of the forearm for a subject performing reach to grasp movements, towards a marker, placed at five (5) different positions in 3D space.

Therefore, we conclude that the muscular co-activation patterns vary significantly not only between different subjects, but also between different reach-to-grasp movements of the same subject (towards different subspaces or different objects placed at specific position), and therefore should be considered and analyzed as subject-specific and task-specific characteristics.

In this section we present some typical specifications for EMG based interfaces and we describe the problem formulation and the methods used for discrimination of different muscular co-activation patterns, associated with different reach to grasp movements (classification) and EMG based motion estimation (regression).

In the aforementioned classification examples we used the random forests technique to discriminate, between different reach to grasp movements in the m-dimensional space of the myoelectric activations, using multiple EMG channels (m is 15 or 16). Its quite typical for EMG based interfaces, a limited number of EMG channels to be available (e.g., due to cost or complexity limitations), or EMG electrodes positioning to be not precise (some EMG channels may be more noisy). Thus, a fundamental question is: “Is it possible to select which EMG channels are the most important? How this features selection can be accomplished?”. With Random Forests we can perform efficient features selection, using their ability to compute the importance score of each feature variable and consequently access the relative importance for all feature variables (e.g., EMG channels).

More precisely random forests use for the construction of each tree, a different bootstrap sample set from the original data. One-third of the samples are left out of the bootstrap sample set (out-of-bag samples) and are not used in the construction of the Nth tree. Feature variables importance, is computed as follows; in every grown tree in the forest, we put down the out-of-bag samples and count the number of votes cast for the correct class. Then the values of a variable m are randomly permuted in the out-of-bag samples and these samples are put down the tree. Subtracting the number of votes casted for the correct class in the m-variable permuted out-of-bag data from the previously computed number of votes for the correct class in the untouched out-of-bag data, we get the importance score of a feature variable m for each tree. The raw importance score for each feature variable m is the average importance score for all trees of the random forest. The random forests feature variable importance calculation procedure, is depicted in Fig. 1.17.

In case that we want to reduce the number of EMG channels used (in this study we have already used 15 and 16 EMG channels), random forests can be initially run with all the variables (EMG channels) and then run once again with the most important variables selected during the first run. For example, we can use the random forests classifier with all 15 EMG channels, compute the feature variables importance and re-solve the classification problem, using the most “important” EMG channels. Before doing so, we present the feature variables importance for the problems of discriminating from EMG signals, reach to grasp movements towards, different subspaces, different objects and different tasks.

In Fig. 1.18 we present the importance plots of different feature variables (EMG channels), for two different cases, subspace discrimination and object discrimination. We can notice that for subspace discrimination, the feature variables corresponding to upper-arm muscles (first 8 EMG channels) appear to have increased importance, while for object discrimination the feature variables corresponding to the forearm muscles (last 8 EMG channels), accumulate most of the importance.