The learning signal driving adaptation in error-based learning is, as the name implies, the error signal between a desired and actual action, as well as the particular way the desired action was missed [106, 125]. The error signal is believed to adapt the motor commands, such that the error decreases in consecutive movements [106]. Wolpert and colleagues reported that in order to adapt to perturbations, the nervous system also estimates the gradient of the error with respect to each motor command component [125]. This means that the motor system needs to have an idea of how components of the motor command attribute to the error, and subsequently how the motor system can reduce the error. Wei and Körding posited that the sensorimotor system might adapt to errors in a nonlinear fashion [124]. They suggested that the sensorimotor system must weigh the information, in this case the error, provided by the uncertainty the information has in the signal. The ideal strategy, they argue, is therefore nonlinear, where small errors are compensated in a linear fashion and large errors would be disregarded. Errors that fall within the expected variance will be adapted for in a fairly linear way, whereas participants showed nonlinear and nonspecific adaptation to single trials containing error signals that exceeded expectation [41, 124].

Temporal processes Smith and colleagues [111] proposed a model in which two parallel temporal processes drive motor adaptation: (1) a fast-acting process that learns and forgets quickly and (2) a slow-acting processes that learns and forgets more slowly. This model is able to explain complex features of motor learning such as spontaneous recovery of learning, savings (relearning of a perturbation or skill is faster than the initial learning), anterograde learning (the ability of a previously learned force field task to reduce the learning rate of a different subsequent task) and even patterns of 24-hour retention [61, 110]. More recent studies suggested that additional learning processes also need to be present to fully explain the temporal evolution of motor adaptation. Lee and Schweighofer [76] proposed a model with a single fast process combined with multiple slow processes, that could explain different types of adaptation tasks. An advantage of such a multi-rate learning model is that it can account for different temporal changes of the sensorimotor system, such as fatigue or injury [76, 125].

Model-based and model-free processes It is likely that multiple processes occur during motor learning, which are often classified as model-based learning processes (e.g., adaptation of the internal model) or model-free learning processes (e.g., use-dependent plasticity and reinforcement learning). For instance, studies have shown that several (model-free) processes occur besides error-based learning (adaptation): use-dependent plasticity [28, 56, 121] and reinforcement learning [56].

It has been shown that repeating a movement in a particular direction does not only reduce movement variability, but also creates a bias toward that direction in future movements [121]. This repetition-induced bias has been termed as use-dependent plasticity [69]. A couple of studies showed that when performing a reaching task in a perturbed environment, adaptation and use-dependent plasticity occur simultaneously [28, 56]. Huang and colleagues used a modified visuomotor rotation paradigm to show that participants, when adapting to the visuomotor rotation, create a bias toward the adapted movement direction [56].

In addition, Huang and colleagues [56] hypothesized that, during a visuomotor rotation adaptation task, hitting a target is a form of implicit reward driving a reinforcement process whereby successful error reduction is associated with the motor commands. They also showed that the model-free reinforcement learning process is independent of model-based learning (adaptation). Combining the model-based adaptation process with the reinforcement process leads to faster relearning (i.e., savings).

Structural learning is a framework to explain the learning-to-learn phenomenom [14, 15]. Structural learning can be considered as learning certain features of a learning task, such that learning of similar tasks is facilitated. Braun and colleagues found support for structural learning by having participants perform reaching movements, during which random visuomotor rotations were imposed. The participants then adapted to a constant visuomotor rotation. They found that being exposed to the random visuomotor rotations facilitated learning in the constant rotation [14]. Braun et al. suggested that training with the random rotations allowed the participants to extract relevant features, or structures, of the task; all tasks were rotations. Structural learning is also consistent within the Bayesian framework, in that it would correspond to learning new prior distributions on the parameters of the perturbation [9, 10, 34].

Although the notions of different learning processes are intriguing, it is still not completely known how the brain performs all these hypothesized actions. Evidence suggests that the cerebellum plays an important role in trial-by-trial error-based learning [8, 26, 29, 117]. More specifically, some studies posit that the cerebellum computes the prediction error-driving adaptation [99, 113]. Patients with cerebellar lesions showed substantial impairment in fast adaptation across different tasks [26, 117]. Brain stimulation studies found that enhanced cerebral activity using transcranial direct current stimulation resulted in faster adaptation [44, 47, 100]. Where different types of adaptation are neurally stored remains an open question [125].

To achieve an increase in performance, such as a reduction in error variability, reinforcement learning can help to find a solution to a movement problem. Reinforcement learning is driven by a reward signal; for instance, the information about the relative success and failure of a movement [41, 125]. In contrast to the error signal in error-based learning, a reward signal does not give information about the direction of required behavioral change [125]. Therefore, reinforcement learning tends to be slower than error-based adaptation. However, when a complex sequence of actions is necessary to achieve a goal, reinforcement learning can be used to explain what actions led to success and which led to failure, whereas error-based learning might be less successful.

Recent research has defined skill acquisition as a shift in the speed-accuracy trade-off function (SAF) [94, 109]. Reis and colleagues argue that defining skill acquisition as a shift in SAF is necessary, otherwise it is not clear how to relate changes in speed and accuracy to a change in skill. For instance, one could reduce execution speed and obtain a higher accuracy by “moving” along the same SAF, which would not reflect a change in skill.

Furthermore, Shmuelof et al. posit that a crucial concept regarding skilled performance is that successful execution and the trajectory kinematics associated with this execution are distinct. This is the case because only the task success is explicitly required, whereas there may be multiple kinematics that reach the desired goal [109]. In an experiment where subjects were instructed to follow a curved path without perturbation using wrist motions, the authors examined changes in the SAF and trajectory kinematics during learning. They found that practicing in restricted speeds led to a global shift of the SAF. Improved performance largely resulted from reduced trial-to-trial variability and increased movement smoothness. The authors propose that motor skill acquisition can be characterized as a slow reduction in movement variability, which is consistent with previous studies [85, 86] but distinct from faster model-based learning, which reduces error in adaptation paradigms.

Optimal feedback control (OFC), as described in Sect. 2.2, could be used to study skill learning [27, 69]. Although OFC has not been used to describe the learning process itself yet, it has been used to explain how we learn to control complex objects with internal degrees of freedom [87], see Fig. 3. For these tasks, there is no simple one-to-one mapping from the hand state to the state of the object (i.e., there are uncontrolled degrees of freedom). During training, participants interacted with the objects and showed improvements in meeting an accuracy criterion even though they had to move faster (i.e., shift in SAF, which is considered to be an improvement in skill). The hand kinematics after training could be described by OFC using a relatively simple cost function. The authors assumed that during training, the participants adapted to the complex dynamics in accordance with a model-based optimization of the cost function [87]. One could speculate that only the model-based optimization part would lead to skill acquisition; however, since the training was not the focus of Nagengast’s study, insufficient data were available. Krakauer and Mazzoni suggested that two processes could occur during training, leading to better performance: convergence to the optimal policy, or improved execution of the control itself. Either of these processes could lead to a shift in SAF [69] and to reductions in movement variability [85, 86].