Assisted Computer Interaction for Users with Weak Upper Limb Motion

Fig. 9.1

The planar pantograph haptic system integrated with a mouse

The device is powered by two precious metal brushed 40 W DC-motors from Portescap (Model 25GT2R82-222E) through capstan cable transmissions which provide a 26:1 gearing ratio with minimum friction and excellent back drivability (back-drivable friction is below 0.2 N). Incremental encoders at the joints monitor the position changes of the device tip, while torque sensors measure the torque at each motor output shaft. The specifications of the device are shown in Table 9.1.

Table 9.1

Specifications of the planar haptic device

Property	Value
Actuator	40 W DC motors
Position resolution	<0.05 mm
Back-drivable friction	<0.2 N
Peak force	26 N
Continues stall force	3.9 N
Workspace	340 × 250 mm

The haptic system end-tip has been interfaced with a mouse to form the assistive interface for the GUI based interactions. To monitor the exchange of forces between the assistive system and user’s hand a force sensor has been installed between the mouse and the haptic device. The cost of the prototype haptic device is 1300€ excluding the cost of the 6DOF F/T sensor (5,000€). This additional cost of the force torque sensor can be significantly reduced by replacing it with a two axis load cell with a cost of approximately 500€. Although a 2DOF load cell is sufficient the 6DOF sensor was used in this study as it was already available in our laboratory.

9.2.2 Device Kinematics

The kinematic configuration of the assistive haptic device is presented in Fig. 9.2. The position vector of the device tip with respect to the base frame O(x _o, y _o, z _o) is denoted by x _t. We can obtain x _tas a function of the active joint angles q_i, i = 1, 2 and the upper link angle q_u:

Fig. 9.2

The kinematic geometry of the haptic device

${\mathbf{x}}_{\mathbf{t}}=\left[\begin{array}{l}\hfill \mathrm{a}+{\mathrm{L}}_1 \cos \left({\mathrm{q}}_1\right)+{\mathrm{L}}_2 \cos \left({\mathrm{q}}_{\mathrm{u}}\right)\hfill \\{\mathrm{L}}_1 \sin \left({\mathrm{q}}_1\right)+{\mathrm{L}}_2 \sin \left({\mathrm{q}}_{\mathrm{u}}\right)\hfill \end{array}\right]$

(9.1)

where L₁, L₂ are the lengths of links and a is the location, along the X-axis, of the proximal joints of the inner links with respect to the base frame. The upper link angle q_u can be determined using the geometry of the five bar mechanism q_u = π − q_d − q_α.

The angle q_d can be computed from the triangle formed by the outer links and the line joining the distal joints of the inner links ${\mathrm{q}}_{\mathrm{d}}= \arccos \left(\frac{d}{2{L}_2}\right)$ while angle q_α is given by ${\mathrm{q}}_{\upalpha}= \arctan \left(\frac{\Delta y}{\Delta x}\right)$ . L₂, d and Δx, Δy represent the length of the distal link, the length of the line joining the distal joints of the inner links and the x, y distances between the distal joints of the inner links respectively. Substituting q_u in (9.1) and directly differentiating the final expression of the forward kinematics yields the device Jacobian which relates the handle tip velocity to the two active joint velocities.

${\dot{\mathbf{x}}}_{\mathbf{t}}=\mathbf{J}\dot{\mathbf{q}}$

(9.2)

The dynamic behavior of the haptic device is finally described by:

$\mathbf{D}\left(\mathbf{q}\right)\ddot{\mathbf{q}}+\mathbf{C}\left(\mathbf{q},\dot{\mathbf{q}}\right)+\mathbf{F}\left(\dot{\mathbf{q}}\right)+{\mathbf{J}}^{\mathbf{T}}{\mathbf{F}}_{\mathbf{d}}=\boldsymbol{\uptau}$

(9.3)

where the newly introduced notations describe: τ the generalized joint torque vector, D(q) the inertia matrix, $\mathbf{C}\left(\mathbf{q},\dot{\mathbf{q}}\right)$ the coriolis/centripetal vector, $\mathbf{F}\left(\dot{\mathbf{q}}\right)$ the friction vector, F _dis the force that the device generates at the end-tip and J ^Tis the transpose Jacobian of the haptic device.

9.2.3 Assistive Control Scheme

The haptic device and its control scheme aims to aid a subject with motion coordination difficulties to interact with GUIs using a mouse by assisting the movements in the trajectory regions where motion is weak. We introduce the assistive control for the haptic device by initially considering the 1DOF linear mechanical system shown in Fig. 9.3.

Fig. 9.3

User hand/device interaction diagram (top) and conceptual 1DOF linear model of the proposed assistive/suppression control law (bottom)

The extension of the concept to the full system will be discussed in the next section. In Fig. 9.3, mass M_h and damper B_h resemble the mass and damping properties of the human forearm and hand segments (mouse motions are primary generated by the movement of the elbow, wrist and finger joints). The user mass/damper block is fixed to the device end-tip through a force sensor which monitors the forces exchanged between the user and the haptic device. The device itself is represented by the mass/damper network denoted by M_d and B_d. The spring/damper arrangement indicated by K_a, B_a is a virtual spring/damper group which is attached to the device tip at its one end point and represents the assistive control network. The remaining notations introduced in Fig. 9.3 are; F_h the force applied by the user, F_d the force provided by the device, F_s the force measured by the sensor, F_a the force generated by the actuator A, X_h the current device end-tip position and X_e the assistive network (K_a, B_a) grounding position which is dynamically updated according to the user effort and intended motion.

9.2.3.1 Assistive Controller Description

Let now assume that the subject’s hand is applying a force F_h through the mouse, while the haptic device is exerting a force F_d. The difference between these two forces is measured as the force sensor F_s = F_h − F_d. To provide the assistive functionality the amplitude and the direction of the assistive force needs to be determined. It is reasonable to use the direction of F_h to determine the direction of the user’s intended motion. The force applied by the user F_h can then be computed from the measured force F_s as follows:

${\mathrm{F}}_{\mathrm{h}}={\mathrm{F}}_{\mathrm{s}}+{\mathrm{F}}_{\mathrm{d}}$

(9.4)

To generate an assistive force in the user’s intended direction of movement, the assistive control network, represented by the spring/damper group (K_a, B_a) is used. One end of this virtual spring/damper system is rigidly linked to the device end-tip, while its second end is free to move; its position X_e is determined using the user’s effort and intended motion. In particular, the desired position X_e of the assistive network grounding end point is updated using the user force F_h. This can be written as:

${\mathrm{X}}_{\mathrm{e}}={\mathrm{G}}_{\mathrm{f}}\ {\mathrm{F}}_{\mathrm{h}}$

(9.5)

where G_f represents the transfer function between the force exerted by the user and the grounding end point position. A possible transfer function G_f which can be used is the simple integrator ${\mathrm{G}}_{\mathrm{f}}=\frac{{\mathrm{K}}_{\mathrm{f}}}{\mathrm{s}}$ . The motivation behind the simple integrator is based on the idea that the accumulation of forces applied by the user can give a good indication/estimation of the user motion intention/direction and therefore it can be used to update X_e towards this direction. Other transfer function candidates include double integrators or other higher order controllers which could in addition permit shaping of the frequency response, e.g. preventing any tremor noise from entering the estimation of the intention of motion. As ataxia subjects do not suffer for this kind of tremor noise a simple integrator was experimentally proved (see section IV) to be adequate in this case.

The perturbation of the free end of assistive spring/damper network will trigger the generation of forces attracting the device end-tip towards the position X_e of the free end-point. In other words the device will generate a force F_d pulling towards X_e, which from the user’s point of view will be perceived as a force assisting his/her movement (the direction of the perturbation of X_e has been selected to coincide with the direction of the user’s applied force F_h). The pulling force is determined by the properties of the virtual assistive spring/damper (K_a, B_a) and is written as:

${\mathrm{F}}_{\mathrm{d}}={\mathrm{G}}_{\mathrm{a}}\left({\mathrm{x}}_{\mathrm{e}}-{\mathrm{x}}_{\mathrm{h}}\right)\kern0.5em \mathrm{with}\kern0.5em {\mathrm{G}}_{\mathrm{a}}={\mathrm{B}}_{\mathrm{a}}\mathrm{s}+{\mathrm{K}}_{\mathrm{a}}$

(9.6)

To exert the assistive force F_d at the device tip we select an actuator control signal as follows:

${\mathrm{F}}_{\mathrm{a}}=\mathrm{u}={\mathrm{F}}_{\mathrm{d}}+{\mathrm{G}}_{\mathrm{d}}{\mathrm{x}}_{\mathrm{h}}={\mathrm{G}}_{\mathrm{a}}\left({\mathrm{x}}_{\mathrm{e}}-{\mathrm{x}}_{\mathrm{h}}\right)+{\mathrm{G}}_{\mathrm{d}}{\mathrm{x}}_{\mathrm{h}}$

(9.7)

where the term G_dx_h with G_d = M_ds² + B_ds is added to compensate for the device dynamics. This pulling force F_d together with the force applied by the user F_h, drives the position of the pair (Haptic device + user’s hand) X_h towards X_e . It can be written:

${\mathrm{x}}_{\mathrm{h}}=\frac{{\mathrm{F}}_{\mathrm{h}}+{\mathrm{F}}_{\mathrm{d}}}{{\mathrm{G}}_{\mathrm{h}}}$

(9.8)

where G_h = M_hs² + B_hs resembles the mass and damping properties of the human hand/finger segments. By injecting F_h, F_d from (9.5) and (9.6) into (9.8) the position X_h can be obtained as a function of X_e.

${\mathrm{x}}_{\mathrm{h}}=\frac{{\mathrm{F}}_{\mathrm{h}}+{\mathrm{F}}_{\mathrm{d}}}{{\mathrm{G}}_{\mathrm{h}}}=\frac{\frac{{\mathrm{X}}_{\mathrm{e}}}{{\mathrm{G}}_{\mathrm{f}}}+{\mathrm{G}}_{\mathrm{a}}\left({\mathrm{x}}_{\mathrm{e}}-{\mathrm{x}}_{\mathrm{h}}\right)\ }{{\mathrm{G}}_{\mathrm{h}}}=\frac{\frac{1}{{\mathrm{G}}_{\mathrm{f}}}+{\mathrm{G}}_{\mathrm{a}}}{{\mathrm{G}}_{\mathrm{h}}+{\mathrm{G}}_{\mathrm{a}}}{\mathrm{x}}_{\mathrm{e}}$

(9.9)

Similarly by substituting X_h and X_e from (9.5) and (9.8) in (9.6) the assistive force F_d developed by the device can be derived as a function of the human applied force F_h.

${\mathrm{F}}_{\mathrm{d}}={\mathrm{G}}_{\mathrm{a}}\left({\mathrm{x}}_{\mathrm{e}}-{\mathrm{x}}_{\mathrm{h}}\right)=\frac{{\mathrm{G}}_{\mathrm{a}}{\mathrm{G}}_{\mathrm{f}}{\mathrm{G}}_{\mathrm{h}}-{\mathrm{G}}_{\mathrm{a}}}{{\mathrm{G}}_{\mathrm{h}}+{\mathrm{G}}_{\mathrm{a}}}{\mathrm{F}}_{\mathrm{h}}$

(9.10)

Substituting G_h, G_a and G_f into (9.9) and (9.10) x_h and F_d can be obtained as a function of the device, the human arm and the assistive network parameters.

9.2.3.2 Stability Analysis and Assistive Gain Selection

The characteristic equation of the transfer functions in (9.9) and (9.10) is

${\mathrm{G}}_{\mathrm{h}}\left(\mathrm{s}\right)+{\mathrm{G}}_{\mathrm{a}}\left(\mathrm{s}\right)={\mathrm{M}}_{\mathrm{h}}{\mathrm{s}}^2+\left({\mathrm{B}}_{\mathrm{h}}+{\mathrm{B}}_{\mathrm{a}}\right)\mathrm{s}+{\mathrm{K}}_{\mathrm{a}}$

This characteristic equation is subject to change due to variations in G_h(s) . This is due to the variability in the mass and damping coefficients of the human arm M_h and B_h. If (B_h + B_a) > 0 and K_a > 0, then G_h(s) is Hurwitz. We may write:

$\frac{{\mathrm{K}}_{\mathrm{a}}}{{\mathrm{M}}_{\mathrm{h}}}={\upomega}_{\mathrm{n}}^2\kern0.5em \mathrm{and}\ \upzeta =\frac{{\mathrm{B}}_{\mathrm{h}}+{\mathrm{B}}_{\mathrm{a}}}{2{\mathrm{M}}_{\mathrm{h}}{\upomega}_{\mathrm{n}}}=\frac{{\mathrm{B}}_{\mathrm{h}}+{\mathrm{B}}_{\mathrm{a}}}{2\sqrt{{\mathrm{M}}_{\mathrm{h}}{\mathrm{K}}_{\mathrm{a}}}}$

(9.11)

where ω_n is the natural frequency for a stable, second-order, closed-loop system and ζ is the damping ratio. Through the control parameters of the assistive spring network (K_a, B_a), we have independent regulation of the natural frequency and damping ratio. We choose a desired $\upzeta =\sqrt{2}/2$ such that the system is underdamped. Therefore from (9.11) we can obtain:

$\upzeta =\frac{{\mathrm{B}}_{\mathrm{h}}+{\mathrm{B}}_{\mathrm{a}}}{2\sqrt{{\mathrm{M}}_{\mathrm{h}}{\mathrm{K}}_{\mathrm{a}}}}\Rightarrow {\mathrm{B}}_{\mathrm{a}}=\sqrt{2{\mathrm{M}}_{\mathrm{h}}{\mathrm{K}}_{\mathrm{a}}}-{\mathrm{B}}_{\mathrm{h}}$

(9.12)

The parameters of the assistive control law (K_a, B_a) are selected by considering the desired system bandwidth and damping ratio. Since the system and controller are used as a mouse based motion assistive device it is sensible to select the (K_a, B_a) gains such that the bandwidth of the system is greater than the maximum motion frequency during mouse based interaction movements (1–2 Hz). For this simulation study a bandwidth of ∼ 5 Hz (ω_n = 30 rad/s) was considered for the 1DOF system presented in Fig. 9.3. The assistive network parameters were determined from (9.11) and (9.12) as K_a = 10 N/cm and B_a30 N s/cm. The mass and damping properties of the human hand/finger were set as M_h = 1 Kg, B_h = 0. The level of damping at the wrist and finger joints during voluntary movements is greatly affected by many factors the most important of which are; the mean level of muscle activation and the angular velocity of the joint [30]. Therefore, the value of the joint damping is a highly uncertain parameter with its maximum level being of an order greater than the minimum. Various studies [32–35] reports damping ratios for the forearm joints (elbow, wrist and finger joints) in the range of 0.1–1.0. By setting B_h = 0 the worst case condition in terms of system stability (hand/finger represented by a pure mass) is applied.

9.2.3.3 Assistive Controller Validation

To test the assistive control scheme, simulation studies were carried out on the 1DOF system shown in Fig. 9.3. The gain parameters used were those reported in the previous section. Here, a step like force input lasting for 0.2 s as shown in Fig. 9.4 (top graph) was used to trigger the 1DOF system.

Fig. 9.4

The simulated responses of the 1DOF assistive control system

The top graph in Fig. 9.4 depicts the resultant perturbation of the grounded end of the assistive spring/damper group X_e and the dynamic behaviour of X_h towards X_e (dashed line). The displacement of X_e triggers the generation of force which pulls the device tip X_h towards the position X_e.

The attracting force F_d plotted against the force applied by the human F_h is shown in the following graph, Fig. 9.4. It increases with the application of the human force F_h to assist the motion and declines as soon as the human force is removed and the device position X_h reaches the target point X_e. The fourth graph in Fig. 9.4 presents the ratio $\frac{{\mathrm{F}}_{\mathrm{h}}}{{\mathrm{F}}_{\mathrm{d}}}$ which can be seen as the reciprocal of the assistive ratio. A high value indicates the absence of assistive forces, while low values reveal their existence. It can be seen that the reciprocal of the assistive ratio is high during the application of the human force, but rapidly declines indicating the rise of the assistive force component.

The Bode diagram in Fig. 9.4 presents the effect of the variation of the mass of the human hand/finger (0.5 Kg < M_h < 2 Kg) to the bandwidth of $\frac{{\mathrm{x}}_{\mathrm{h}}}{{\mathrm{x}}_{\mathrm{e}}}$ dynamics demonstrating a bandwidth of ∼ 5 Hz (30 rad/s) for a nominal mass of M_h = 1Kg . In Fig. 9.5 the variation of the damping ratio of $\frac{{\mathrm{x}}_{\mathrm{h}}}{{\mathrm{x}}_{\mathrm{e}}}$ dynamics is presented as a function of the mass of the human hand/finger M_h. These simulation results demonstrate the potential of the proposed scheme as an assistive controller when triggered by the application of the human force.