2.1 Participants

Eleven healthy male subjects with a mean age of 23 years (ranging from 21 and 25 years) participated in the experiment. None of the subjects had any research experiences in human motor control. Experimental procedures of the study were in accordance with the Declaration of Helsinki, and approved by the Ethics Committee on Human Experiments at Graduate School of Engineering Science, Osaka University. All subjects gave written informed consent prior to their inclusion in the study.

2.2 Outline of the experiments

Virtual balancing involved balancing a simulated single inverted pendulum implemented in a digital signal processor (DSP) using an EMG-based human-computer interface (Fig. 1). Subjects were asked to stand upright normally on a force platform barefoot. Motion of the virtual pendulum was displayed in real-time on an oscilloscope screen located in front of subjects as a vertically moving horizontal bar representing the tilt angle of the pendulum. Upward and downward motions of the bar represented forward and backward tilts of the pendulum, respectively. Subjects were instructed to keep the pendulum upright as much as possible under the continuous visual feedback information about the motion of the pendulum. Activation levels of two antagonist ankle muscles of subjects, namely, Tibialis Anterior (TA) and Medial Gastrocnemius (MG) were monitored by electromyography (EMG). The EMG signals were fed into the DSP to obtain integrated electromyograms (iEMGs) of those muscles, which were further transformed into simulated antagonist muscle torques for actuating the pendulum in real-time. Activations of TA and MG muscles generated the simulated muscle torques to pull the pendulum forward and backward, respectively, as in human standing. Subjects modulated their muscle activation levels by altering the tilt angle of their standing body. Subjects were instructed that no joint movements of upper and lower limbs, except the ankle joints, were allowed. Thus, modulation of the body tilt angle was achieved by dorsal and plantar flexions of the ankle joints, inducing small voluntary body sway of subjects in the anterior-posterior direction. It was expected that, when the pendulum tilted forward, subjects also tilted forward to increase the activation level of MG so that the pendulum could be pulled backward, whereas when the pendulum tilted backward, subjects also tilted backward to activate TA muscle so that the pendulum could be pulled forward, implying that subjects might keep track of a sequence of falling motions of the pendulum for balancing the pendulum.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. An EMG-based human-computer interface used for virtual balancing.

EMGs derived from TA and MG muscles of subjects were fed into a DSP, which were transformed into active torques for balancing a virtual inverted pendulum in the DSP. A vertically moving horizontal bar representing the tilt angle of the pendulum, as a visual feedback information for subjects, was displayed on an oscilloscope screen in front of subjects. EMG signals, force-platform-based CoP motion of subjects and tilt angle of the pendulum were recorded by a PC.

https://doi.org/10.1371/journal.pone.0062956.g001

2.3 Virtual inverted pendulum

The virtual inverted pendulum was allowed to rotate around a pin joint at the distal end of the pendulum. The joint of the pendulum corresponds to human ankle joints. Dynamics of the pendulum was modeled by the following equation of motion in the sagittal plane.(1)where , and represent the moment of inertia around the joint, the tilt angle from the upright position, and the total torque acting at the joint, respectively. , , and are the mass, the gravitational acceleration, and the length between the joint and the center of mass of the pendulum, respectively. The term approximates the gravitational toppling torque for small . Values of these parameters were similar to those of a typical human body, and fixed as in Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Parameter values of the virtual inverted pendulum.

https://doi.org/10.1371/journal.pone.0062956.t001

The total torque was modeled as , where was the passive torque determined by a torsional viscoelastic element at the joint, and the active torque determined by the muscle activations derived from subjects. The passive torque was implemented as , where and are the passive elastic and viscous coefficients. The active torque was determined by the difference between the antagonist torques as , where and represent, respectively, the forward and the backward bending torques. and were determined based on the iEMG signals derived from TA and MG muscles of subjects, respectively (see below).

The state space representation of Eq. 1 is as follows.(2)where . We also use the following abstract formulation of Eq. 2.(3)where , and is the active control torque vector. The term defines the vector field of the non-actively-actuated pendulum with .

According to experimental evaluations of the passive ankle visco-elasticity during human quiet standing [54], [55], we set as Nm/rad and Nms/rad. (The value of used here might be several times larger than human passive viscosity. However, it was too difficult for most of the subjects to stabilize the pendulum for a smaller value of .) Thus, , and the upright state is an unstable equilibrium of saddle type, with stable and unstable manifolds when () [32]. For the pendulum parameter values used here, the stable and unstable manifolds of in the - plane are given, respectively, by the straight lines passing through the origin;(4)

and(5)where

(6)Thus, time constants of the stable and the unstable dynamic modes are 0.65 s and 0.78 s, respectively. Solutions (trajectories) generated by the vector field of in the - plane exhibit hyperbolic curves that approach the saddle point at along the stable manifold and fall away from the saddle point along the unstable manifold.

If this mechanical body plant is actively actuated by a continuous delay feedback control that approximates the neural feedback control with a neural transmission delay, a non-negligible risk of the delay-induced instability arises. For example, if we assume a simple proportional and derivative (PD) control with no delay, the unstable saddle point can easily be stabilized asymptotically for a proportional gain greater than 0.2 with any positive derivative (velocity) gain. However, if a typical neural transmission delay of 0.2 s is introduced for the PD feedback control, the upright equilibrium can be destabilized by a Hopf bifurcation [14], [56]. Indeed, for small values of derivative gain, the proportional gain of 0.2 is a critical value, meaning that the proportional gain greater than destabilizes the upright equilibrium. See [44] for safety parameter regions of the gains in the delay feedback control, and confirm that the feedback gains (thus the joint impedance) must be carefully tuned for the stabilization, if the continuous feedback control is employed. A reaction time to visually supplied falling motions of the virtual inverted pendulum in this study corresponds to the feedback time delay, and in the later section, we show that it was much larger than 0.2 s, implying that stabilization of the upright equilibrium by the use of continuous feedback control is not easy.

2.4 Setup of the EMG-based human-computer interface

EMG signals were monitored from TA and MG muscles of both legs using bipolar surface electrodes separated by 1 cm and a biosignal amplifier (BA1104, TEAC, Inc., Tokyo). Body sway of subjects was recorded as the vertical ground reaction force (), the moment around the mediolateral axis () and the moment around the anterior-posterior axis () using a force platform (Advanced Mechanical Technology, Inc., USA). represented motions of Center of Pressure of subjects in the anterior-posterior direction, referred to as CoP. The EMG signals only from the right leg were fed into the DSP (DSK6713IF-A, Hiratsuka Engineering Co. Ltd., Japan) with a sampling frequency of 1 kHz through a 16 bit A/D converter mounted on the DSP, and numerically rectified in the DSP and then processed by the second order Butterworth filter with a cutoff frequency of 12 Hz, yielding the iEMG signals in real-time. The EMG signal from the left leg was not used for the active control of the pendulum, about which subjects were not informed.

In the DSP, dynamics of the virtual inverted pendulum with the active torque was numerically integrated in real-time using the forward Euler method with a time step of 1 ms. Tilt angle of the pendulum was displayed in a range between −0.1 to 0.1 rad, using the analog oscilloscope placed in front of subjects through a 16 bit D/A converter mounted on the DSP. The iEMG signals, data from the force platform and the tilt angle of the pendulum were recorded by a PC with a sampling frequency of 1 kHz through an 14 bit A/D converter (Keyence, Inc., Japan).

2.5 Active torques

Each active torque used for actuating the pendulum was determined as a constant multiple of the iEMG signal. Denoting iEMG of TA and MG muscles as and , respectively, we determined the active torques as(7)where and represent the constant gains. These gains were determined individually for each subject prior to the balancing task, because amplitudes of EMG signals for an identical body tilt angle varied across subjects. See Text S1 for details of how we determined the gain constants.

2.6 Protocol

The balancing trial started with the initial condition of the pendulum at rad and rad/s. When the pendulum's tilt angle departed from a range of rad, we judged that the pendulum fell down, and counted this as a failure event in the task. If subjects succeeded in keeping the pendulum upright with no failure for 60 s, the trial was considered as a success, or otherwise a failure. After every failure event in a trial, the task was suspended for 3 s, whereby the tilt angle of the pendulum was reset to the initial condition, and then subjects restarted the task until the total performance time span, neglecting the suspended periods, reached 60 s. Initially, all subjects performed twenty trials. If three or more trials were successful, the session was terminated. Otherwise, subjects continued to perform trials until three successful trials were completed, or a maximum of forty total trials was reached.

2.7 Data analysis

We examined whether subjects could balance the virtual pendulum by finding appropriate activations of the antagonist ankle muscles through trial and error during the task. In particular, we were interested in examining whether activations of the antagonist muscles were continuous or intermittent. Moreover, we characterized sway motions of the pendulum using the phase space analysis in the - plane and the power spectral analysis.

3.1 Typical examples and classification of strategies

Figure 3 shows typical examples of failed and successful trials in three different subjects (Subjects 4, 2 and 9). Indeed, they exhibited representative behaviors for three types of stabilizing strategies (continuous type, one-sided intermittent type, and two-sided intermittent type). Let us introduce those types here for readability of the result section. Through our analysis described in this section, the one-sided and two-sided intermittent types were characterized as the intermittent control, and the continuous type as the continuous impedance-like control. Subject 4 was classified into the continuous type, and Subjects 2 and 9 were classified into the one-sided and two sided intermittent types, respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Typical data of failed and successful trials.

In each panel, top, second, third and bottom traces show, respectively, tilt angle of the virtual inverted pendulum , CoP of subjects, active torque and active torque . A and D: Failed (#1) and successful (#14) trials in Subject 4 (continuous type). B and E: Failed (#1) and successful (#27) trials in Subject 2 (one-sided intermittent type). C and F: Failed (#1) and successful (#29) trials in Subject 9 (two-sided intermittent type).

https://doi.org/10.1371/journal.pone.0062956.g003

All of those representative subjects failed to balance the pendulum in the first trial. Subject 4 (continuous type) exhibited only 7 failure events even in the first trial, during which TA was almost continuously activated. Activations of MG were short-lived, but accompanied with a small tonic component (Fig. 3A). Contrastingly, Subjects 2 and 9 (one-sided and two-sided intermittent types) experienced, respectively, 17 and 20 failure events in the first trial (Figs. 3B and C). For each failure event, the pendulum started falling backward (or forward) from the initial condition for a period of time, during which those subjects did not react to the falling motion. After a while, a single delayed impulsive, short-lived for about 1.0 s, TA (or MG) activation from the null activation level was generated in response to the backward (or forward) falling motion until the pendulum reversed the direction and moved across the upright position, where the impulsive activation was terminated. Another delayed impulsive activation of MG (or TA) was then generated to brake the forward (or backward) motion, but it could not prevent the pendulum from falling, leading to a failure event.

Subject 4 (continuous type) succeeded for the first time by the 10-th trial, which was followed by consecutive successful trials. Subjects 2 and 9 (one-sided and two-sided intermittent types), on the other hand, failed repeatedly, but eventually succeeded in three trials (Table 2). For each subject, sway angle of the virtual inverted pendulum during successful trials was maintained around the upright position (). However, they exhibited qualitatively different sway patterns. In Subject 4 (continuous type), sway of the pendulum, and also CoP sway of the subject, varied within relatively small ranges (std of 0.025 rad). Moreover, they involved high frequency oscillatory components. Contrastingly, in Subjects 2 and 9 (one-sided and two sided intermittent types), the pendulum as well as the CoP of the subjects exhibited slow sway. Amplitude of the slow sway of in Subject 2 (one-sided intermittent type) was small (std of 0.015 rad), but that in Subject 9 (two-sided intermittent type) was large (std: 0.036 rad). Sway amplitudes of the pendulum (std of ) in successful trials were significantly smaller than those in early stage of learning in most of the subjects, including other subjects than those three representative subjects ( or ), except Subjects 1 and 9 (two-sided intermittent type). Interestingly, average amplitudes for successful trials in Subjects 1 and 9 were larger than those in the remaining subjects.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary of performance of all subjects.

https://doi.org/10.1371/journal.pone.0062956.t002

The active torques ( and ) generated by the three representative subjects during successful trials were also different from each other. The active torques of Subject 4 (continuous type) exhibited tonic and high frequency bursts as in the sway waveforms. MG muscle of Subject 2 (one-sided intermittent type) was almost silent, and the activation of TA muscle was more phasic in comparison with Subject 4 (continuous type), and its amplitude was small. Both muscles of Subject 9 (two-sided intermittent type) exhibited sparse and intermittent bursts.

As described here, those preliminary observations across all subjects suggested that there might be three types of intervention strategies; high-frequency, almost continuous interventions as in Subject 4 (continuous type), medium frequency interventions only by one of the two muscles as in Subject 2 (one-sided intermittent type), and low frequency, intermittent interventions as in Subject 9 (two-sided intermittent type).

Sway dynamics of the pendulum were characterized by trajectories in the - phase plane (Fig. 4). Each trajectory spanning 60 s was segmented and classified into four different groups of segments. Segments of a trajectory in each group were accompanied by a group-specific combination of activation and/or inactivation of TA and MG muscles: Group-1) both TA and MG were inactivated, referred to as the inactive segments, Group-2) TA was inactivated but MG was activated, referred to as the MG-active segments, Group-3) TA was activated but MG was inactivated, referred to as the TA-active segments, and Group-4) both TA and MG were co-activated, referred to as the co-active segments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Sway trajectories of the pendulum in the - phase plane.

Configuration and data used for panels A to F were the same as those in Fig. 3. Each trajectory was segmented into four different groups based on the associated combinations of activations in TA and MG muscles. Back curve segments: Both TA and MG were inactivated (Inactive). Blue segments: TA was inactivated but MG was activated (MG-active). Red segments: TA was activated but MG was inactivated (TA-active). Green segments: Both TA and MG were co-activated (Co-active). Dashed and dotted lines represent the stable and the unstable manifolds of the saddle point defined by Eqs. 4 and 5, respectively, for the virtual pendulum with no active torque.

https://doi.org/10.1371/journal.pone.0062956.g004

Trajectories of failed trials in Subject 4 (continuous type) exhibited circular motions around the origin of the plane (Fig. 4A). The associated muscle activation patterns were simple. TA was activated almost continuously when the pendulum was tilted backward to pull the pendulum forward (TA-active segment). When the pendulum was pulled forward across the upright position, TA was inactivated and MG was then activated to pull the pendulum backward (MG-active segment). Transition interval from an inactivation of TA (or MG) to the successive activation of MG (or TA), where both TA and MG were inactivated or co-activated (inactive or co-active segment), was small. Successful trials inherited this feature, but the motions were bounded more tightly around the origin than those in the failed trial (Fig. 4D). Transition intervals were small also in the successful trials, leading to more continuous muscle activations and short time intervals with no muscle activations.

On the other hand, trajectories of Subjects 2 and 9 (one and two-sided intermittent types) in failed trials could be characterized differently by repeated backward and forward slow falling motions along the unstable manifold of the saddle point (Figs. 4B and C). The corresponding segments of trajectories were associated with “slow natural motions”during which those subjects of intermittent types did not activate neither TA nor MG muscle (inactive segments). It was remarkable that this feature of slow natural motions was inherited by their successful trials, and thus the pendulum was not actively actuated for long periods of time (Figs. 4E and F). In successful trials of Subject 2 (one-sided intermittent type), the trajectory circulated almost only in the left-half plane, in which the inactive segments of hyperbolic arcs appeared intermittently (Fig. 4E). Each inactive segment represented a forward motion approaching the origin (the upright position) along the stable manifold and then falling backward along the unstable manifold. In Subject 9 (two-sided intermittent type), the inactive segments formed the upward and downward convex-hyperbolic curves (Fig. 4F) approaching the origin along the stable manifold in one side of the plane, passing across the upright position, and then falling along the unstable manifold in the other side of the plane, suggesting that these natural motions in the inactive segments were governed by the saddle-type vector field of defined in Eq. 3. Moreover, the trajectory circulated within each of left and right sides of the plane, where inactive segments of hyperbolic arcs appeared intermittently as in the case of Subject 2 (one-sided intermittent type). This implied that there were two oscillatory dynamics at left and right sides of the plane, and the state of the pendulum was hopping between them.

Based on the observations described here, we classified nine subjects into either continuous, one-sided intermittent, or two-sided intermittent type. Subjects 1 and 9 were classified into the two-sided intermittent type. Subjects 2, 3, 5 and 8 were classified into the one-sided intermittent type. Subjects 4, 7 and 11 were classified into the continuous type. Those classifications were performed based on several quantitative indices. In this sequel, we describe detailed dynamics during the motor-task with those indices, by which we characterized each type of the control strategy adopted by the subjects.

3.2 Activation, inactivation and co-activation of the muscles

Time intervals during which TA and/or MG were activated and inactivated were analyzed for all failed and successful trials of nine subjects. For each trial, the whole time span (60 s) was divided into four groups of intervals based on the group-specific combinations of activation and/or inactivation of TA and MG muscles. As in Fig. 4, those four groups were 1) the inactive intervals, 2) the MG-active intervals, 3) the TA-active intervals, and 4) the co-active intervals. Average total duration of each interval group was analyzed over the first five (F5) trials and also over all successful trials for each subject to characterize how the subjects adapted to the task (Fig. 5). Note that the sum of average durations of inactive, TA-active, MG-active, and co-active intervals was equal to 60 s.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 5. Average durations of inactive, TA-active, MG-active and co-active intervals.

See caption of Fig. 4 for definitions of four groups of intervals. Each duration was averaged separately over the first five trials (F5) and the successful trials (S) for each subject. Black bars: Inactive durations. Dark gray bars: MG-active durations. Gray bars: TA-active durations. White bars: Co-active durations.

https://doi.org/10.1371/journal.pone.0062956.g005

In all subjects, average inactive durations of the successful trials decreased by a few seconds up to 10 s from those of the F5 trials, and durations with muscle activations increased by the corresponding amounts. However, in Subjects 1 and 9 (two-sided intermittent type), and Subject 5 (one-sided intermittent type), the duration of each group in the successful trials was roughly the same as that in the F5 trials, suggesting that subjects did not necessarily optimize overall frequency of interventions.

3.3 Similarity to the saddle-type vector field

Direction cosine (DC) defined by Eq. 8 was calculated to quantify similarity between a phase plane trajectory of the virtual pendulum during each successful trial and the theoretical saddle-type vector field of the pendulum in the absence of active feedback torque. DC-values for failed trials, in particular for F5 trials, were not considered, because DC-values were always close to unity during falling motions along the unstable manifold. If the pendulum moves along the saddle-type vector field in a period of time during a trial, the DC-value in that period of time becomes close to unity. Fig. 6 exemplifies how the DC-value was modulated along time during successful trials in the representative of three types (Subjects 4, 2 and 9).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 6. Time-courses and distribution histograms of Direction Cosine (DC) during successful trials.

A. Trial 14 of Subject 4 (continuous type). B. Trial 27 of Subject 2 (one-sided intermittent type). C. Trial 29 of Subject 9 (two-sided intermittent type).

https://doi.org/10.1371/journal.pone.0062956.g006

We analyzed the total period of time during which the DC-value stayed above 0.8 (i.e., between 0.8 and 1.0) for each of nine subjects. Fig. 7 shows the ratio of the total period of time with DC0.8 to the whole time span, referred to as the DC-Ratio, averaged over all successful trials for each subject. For example, the average DC-ratios for Subjects 4 of the continuous type, Subject 2 of the one-sided intermittent type and Subject 9 of the two-sided intermittent type in their successful trials were 0.16, 0.46 and 0.46, respectively. Fig. 7 also allows us to associate the DC-Ratios with the durations of inactive intervals. In particular, defining the ratio of the inactive interval duration to the whole trial time span, referred to as Inactive-Ratio, we confirmed that the values of DC-Ratio were in agreement with the corresponding values of Inactive-Ratio. That is, the larger the Inactive-Ratio, the larger was the DC-Ratio.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 7. Multiple indices characterizing sway of the virtual inverted pendulum for each subject.

In each panel, values of five indices are plotted as thick black bars. From the left-most index, they represent the ratio of inactive interval to the whole trial time span (Inactive-Ratio) denoted as InactR, the ratio of time interval with DC to the whole single trial time span (DC-Ratio) denoted as DCR, the ratio of the number of off-points (an off-point is a state point at a time instant when an inactive interval began) located in specific areas of the 2nd and the 4th quadrants of the phase plane to the total number of off-points (Q-Ratio) denoted as QR, the average of distance between off-points and the -axis in the phase plane denoted as Dis (15 times of Dis-value for scaling: 15Dis), and the average standard deviation of -coordinates of off-points denoted as SD (15 times of SD for scaling: 15SD). See text for detailed definitions of indices. Thick black bars are for average values of the indices for successful trials. Thin gray bars are the corresponding values for F5 trials. See text for the meaning of configuration of panels for nine subjects.

https://doi.org/10.1371/journal.pone.0062956.g007

3.3.1 DC-ratio for the continuous type.

In the subjects of continuous type, the DC-value showed high frequency oscillations, and was distributed almost uniformly in a range between -1 and 1 (Fig. 6A), indicating that the vector field governing the motion of the pendulum in subject of the continuous type was not saddle-type. This was in agreement with the fact that the subjects of continuous type showed the short inactive durations (Fig. 5). The average DC-Ratio for the subjects of continuous type was 0.22 s. See Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Changes in average values of the five indices from F5 trials to successful trials through learning for each subject and for average of each type.

https://doi.org/10.1371/journal.pone.0062956.t003

3.4 Inactivation timings of the muscles

In order to characterize the strategies adopted by the subjects, we analyzed changes in the timings when the antagonist muscles were both inactivated. To this end, we specified a location of the pendulum's state point in the - plane, referred to as the off-point, at every instant when both TA and MG were inactivated simultaneously. In other words, we specified a sequence of switching-timings from MG-active, TA-active or co-active intervals to inactive intervals. Fig. 8 shows locations of the off-points for failed and successful trials in the representatives of thee types (Subjects 4, 2 and 9). We examined how distributions of the off-points changed through the motor learning, which characterized the state-dependent timing-strategies adopted by the subjects. Since the activation patterns of the subjects of three types were characterized differently even in the first trials, the off-points were also distributed differently in the first trials (Figs. 8A, B and C).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 8. Distributions of off-points in failed and successful trials.

An off-point was defined as a state point at a time instant when an inactive interval began. Configuration and data used for panels A to F were the same as those in Figs. 3 and 4. Q-Ratio (Quadrant-Ratio) used to characterize the timing strategy of subjects counted the number of off-points located in the gray triangular areas defined in the second and the fourth quadrants.

https://doi.org/10.1371/journal.pone.0062956.g008

The off-points were distributed widely and almost uniformly in the subjects of continuous type (Fig. 8A), reflecting the fact that they tried to keep the upright position of the pendulum by the high frequency bursts of activations. On the other hand, the off-points were located mainly in the first and third quadrants in the subjects of one and two-sided intermittent types (Figs. 8B and C), reflecting the fact that their delayed response to a falling motion in one direction (forward or backward) was short-lived and terminated when the state point started another falling motion in the other direction along the unstable manifold.

Through learning, the distribution of off-points changed and tended to be close to the -axis in the subjects of continuous type (Fig. 8D). This change was characterized by the distance between each off-point and the -axis. We referred to the distance index as Dis. Values of Dis index for the successful trials in the subjects of continuous type were small (Fig. 7 and Table 3), implying that the activated muscle switched immediately before or after the sign of altered. More specifically, the activated muscle switched from MG to TA at an instant when the pendulum moved from forward to backward position, and also switched from TA to MG at an instant when the pendulum moved from backward to forward position, by which the subjects could keep the pendulum's position as close to the upright as possible.

In the subjects of one and two-sided intermittent types, on the other hand, the off-points did not necessarily tend to be close to the -axis. Instead, they moved into the second and/or the fourth quadrants (Figs. 8E and F). Since the off-points in the fourth quadrant were close to the stable manifold of the saddle point of the pendulum in the absence of active control, the state point with the forward-tilted posture and backward velocity moved toward the upright saddle point transiently if no active torques were present. Similarly, the state point in the second quadrant with the backward-tilted posture and forward velocity also moved close to the stable manifold, resulting in the transient approach toward the upright saddle point with no help of muscle activations. These changes in the off-point distribution were characterized by the ratio of the number of off-points located in the triangular areas defined in the second and the fourth quadrants to the total number of off-points. We referred to this ratio as Q-Ratio (Quadrant-Ratio). We defined the two triangular areas such that points in the areas satisfied the following two inequalities; and , where we set . See gray colored triangular areas in Fig. 8. As shown in Fig. 7, values of Q-Ratio for most of the subjects in one and two-sided intermittent types were large. Since these triangular areas did not include the neighborhood of -axis, off-points distributed close to the -axis, as in the subjects of continuous type, did not contribute much to heightening the Q-Ratio value. Based on this, we used the Q-Ratio value to discriminate the timing-strategy adopted by the subjects of one and two-sided intermittent types from those by the continuous type.

In the successful trials of the subjects of two-sided intermittent type, about half of the off-points located in the triangular areas of the second or the fourth quadrants, and thus the Q-Ratio was about 0.5 (Fig. 7 and Table 3). Since the off-points were distributed relatively widely within the second and the fourth quadrants, the Dis-values of those off-points were also large. Thus, large values of Q-Ratio and Dis indices characterized the timing-strategy of the two-sided intermittent type (Fig. 7 and Table 3).

In the subjects of one-sided intermittent type, the off-points were localized in a small area close to the stable manifold at the second (or the fourth) quadrant. Thus, values of Q-Ratio for the subjects of one-sided intermittent type were quite large (Fig. 7 and Table 3). As confirmed by Fig. 8E, the state point, departed from the off-point, moved forward and approached the upright saddle point along the stable manifold in the second quadrant, and then fell away backward from the saddle point along the unstable manifold in the third quadrant. The activation of TA muscle braked and reversed the falling motion, which brought the state point back to the small area in the second quadrant, generating the slow cyclic trajectory located at the left-half phase plane. In this way, the subjects of one-sided intermittent type exhibited the transient stable motion generated by the saddle-type vector field as in the two-sided intermittent type.

Average Dis-values of the localized off-points for the subjects of one-sided intermittent type were larger than those of the continuous type. However, the difference was not significant, and thus, Dis-value alone could not uniquely characterize dynamics of the pendulum for the one-sided intermittent type, because for some subjects of this type, off-points localized at small areas closer to the -axis as in the continuous type. So, we also quantified a degree of localization of off-points using the standard deviation of -coordinates of the off-points, and referred to this index as SD. Because of well-localized off-points in the subjects of one-sided intermittent type, their SD-values were quite small. In this way, we considered that the timing-strategy of the one-sided intermittent type could be characterized by the quite large value of Q-Ratio and the quite small value of SD-value (Fig. 7 and Table 3).

3.5 Power spectral analysis and energy efficiency

Power spectral density (PSD) was estimated for every successful trial. Slopes (scaling factors) at the low and the high frequency bands (LF and HF) of the log-log plotted PSD were calculated to characterize the double power-law behavior in sway of the virtual pendulum. Figs. 9A, B and C show PSDs of the representative subjects of three types.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 9. Power spectral density (PSD) functions of sway dynamics () of the pendulum during successful trials.

A: Trial 14 of Subject 4 (continuous type). B: Trial 27 of Subject 2 (one-sided intermittent type). C: Trial 29 of Subject 9 (two-sided intermittent type).

https://doi.org/10.1371/journal.pone.0062956.g009

Table 4 summarizes average slopes at LF and HF bands that were examined separately over the subjects of each type. The average values of slope at LF band for the subjects of two-sided intermittent, one-sided intermittent and continuous types were −0.99, −1.31 and −0.41, respectively. The slopes for two and one-sided intermittent types were significantly steeper than that for the continuous type (). The absolute values of slope at LF band in the intermittent types were close to the corresponding scaling exponent of () reported by Collins and De Luca [52] in their two-point correlation functions and by Asai et al [32] in their simulation of the intermittent control model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Slopes of PSD at LF and HF bands for successful trials and power consumption [W] for F5 and successful trials for each subject and for average of each type.

https://doi.org/10.1371/journal.pone.0062956.t004

The average values of slope at HF band in the two-sided intermittent, one-sided intermittent and continuous types were −5.92, −5.99 and −5.63, respectively. There were no significant differences in the slope of the HF band between any two types. The absolute values of slope at HF band in all of three types were about two times larger than the scaling exponent of () reported by Collins and De Luca [52]. However, the value of about 6.0 was in agreement with the corresponding exponent reported by Asai et al [32] and Nomura et al [53] in their simulation of the intermittent control model.

The steep slopes at LF band for the subjects of one and two-sided intermittent types implied that the long-term-correlated fluctuations were included in the sway motion of the pendulum controlled by the subjects adopted the intermittent control. This was consistent with the fact that sway of in the subjects of those two types showed slow oscillations, whereas that in the subjects of continuous type did not exhibit slow oscillations, but included high frequency components.

Notice that the average of total power for the continuous type was significantly smaller than that of those for the intermittent types, reflecting the fact that the amplitude of slow oscillation in the continuous type was smaller than those in the intermittent types.

We estimated mechanical energies (powers) consumed by the active torques during F5 and the successful trials (Table 4). For F5 trials, mean powers for the two-sided intermittent, one-sided intermittent and continuous types were W, W and W, respectively, and, for the successful trials, they were W, W and W, respectively. In most subjects, the power was reduced through the learning (but significantly only in three subjects). Moreover, balancing during F5 as well as successful trials for the intermittent types was more energetically efficient than that for the continuous type (). Thus, we concluded that stabilization by the intermittent activations of antagonist muscles for the intermittent types was more energetically efficient than that by the continuous activations for the continuous type.

4.1 Intermittent use of the stable manifold of saddle-type equilibrium

We showed that the majority of subjects did not adopt the impedance control. Instead, they acquired a discontinuous, intermittent control strategy, in which both of two antagonist muscles were inactivated simultaneously and intermittently, leading to the intermittent appearance of periods of time during which the pendulum was not actively actuated. Let us refer to those periods of time as the “off-periods,”and the remaining periods with muscle activations as the “on-periods.”Based on our analysis of behavioral data, we revealed that a specific type of the intermittent control, among different versions of the intermittent control, was adopted by the majority of subjects through motor learning. In particular, for those subjects, we demonstrated that each of the off-periods began at a specific timing when the state of the pendulum visited a safe neighborhood of the stable manifold of the upright saddle-type equilibrium exhibited by the pendulum in the absence of active control, i.e. during the off-periods. Dynamics of the virtual pendulum described here are consistent with the concept of “gentle taps”for the subjects of the intermittent types, and “brute force”for the subjects of the continuous type, adopted for stabilizing unstable loads [59].

The behavior exhibited by the subjects of (one-sided and two-sided) intermittent types was characterized by the long interval of off-periods occupying nearly half of the task-execution time and the distribution of the state point in the phase plane at every onset of off-period. In this way, these subjects selected not to intervene in the control-free natural motion of the pendulum, during which the state point of the open-loop pendulum close to the stable manifold transiently approached the unstable upright saddle point with no active effort. Of course, after a while, the pendulum started falling away from the saddle point, because the upright saddle point was unstable. Thus, in response to the falling motion, the subjects activated either TA or MG muscle for a short interval of an on-period, by which the velocity of the pendulum, not the tilt angle , altered the sign, resulted in the state point of the pendulum close to the stable manifold again. Repeating this process could lead to the oscillatory dynamics, typically limit cycles or chaotic oscillations [42], [60], [53]. In this regard, the dynamics observed for the one-sided and two-sided intermittent types were associated with the bistability of the limit cycle dynamics. Dynamics of the one-sided intermittent type correspond to one of the two bistable limit cycle oscillations, located at either left or right half of the phase plane, and those of the two-sided intermittent type to the hopping dynamics between the two limit cycle attractors.

Notice that a role played by the short-lived activations of TA and MG muscles during the on-periods in the subjects of the intermittent types was to alter the sign of the velocity, not the angle. This is consistent with a previous study demonstrating an importance of velocity information for stabilizing upright standing [61]. However, this importance has been interpreted, so far, in terms of continuous-time derivative feedback control, whereas we reconsidered it from a discontinuous-time perspective. As we discuss later in this section, it is worth mentioning that the acceleration information can also be important for the stabilization as demonstrated by Insperger et al [44].

It has been shown that the intermittent control model that utilizes the stable manifold during the off-period can establish robust and flexible stability of the upright posture for single [32] as well as double inverted pendulums [33] actuated at the ankle and/or hip joints, where the intermittent control model alternates between two types of unstable dynamics, depending on the time-delayed physical state of the body pendulum: (1) one mode is characterized by the fact that the active torque is switched off when the pendulum state is located near the stable manifold of the saddle instability; (2) in the other mode, which operates in the remaining time periods, the state is driven by the delayed feedback controller that induces the delay-induced instability. It is remarkable that although both modes are unstable if any of them was adopted permanently, their combination with appropriate switching timings is stable and robust.

In this sequel, we discuss our results in relation to the other possible strategies with time-continuous and discontinuous controllers.

4.2 Stabilization by time-continuous controllers

Although the majority of subjects adopted the intermittent control strategy, one third of subjects, classified as the continuous type, achieved balancing by making their muscle activations look as much like co-activated as possible by swaying their body twitchily and continuously. In those subjects, duration of the off-periods was short. This suggested that they adopted a continuous control strategy similar to the impedance control for stabilizing the pendulum. However, since co-contractions of TA and MG muscles were relatively hard to be performed, the control was inevitably to be reactive (responding to the tilt angle of the pendulum), as opposed to a preprogrammed feedforward control mechanism such as that seen during voluntary arm reaching movements [17], [18]. Indeed, it was more like a bang-bang or a sliding mode control [62], [63], in which two antagonist control commands with opposite actions were switched alternately at the line of where the sign of the tilt angle was altered.

The outcome of stabilization based on the continuous strategy was surprising, because a continuous and reactive feedback control with a large delay could easily induce instability. However, appropriate tuning of gains for a continuous delay feedback control could avoid instability. See Insperger et al [44] for the stability region in the gain-parameter-space of a continuous delayed proportional and derivative (PD) feedback control of the inverted pendulum. Moreover, they also showed that stabilization by the continuous delayed feedback in the inverted pendulum becomes much easier if a continuous acceleration feedback control is introduced and utilizes a delayed PDA controller. This suggests another possibility that the subjects of our continuous type might utilize the acceleration information for stabilizing the pendulum in the framework of the continuous control.

Yet another candidate for the continuous control strategy is noise-induced stabilization with a state-dependent multiplicative motor noise [23], [25], because variability of motor noise increases with the size of the active motor control signal [64]. Note that, in this case, the controller actuates the pendulum continuously and stochastically, but the resultant dynamics can exhibit intermittency that arises when the delay-affected system is tuned at, or near, the edge of stability and a critical control parameter is stochastically forced back and forth across the boundary [23], in which the term of intermittent is in the sense of intermittency used in statistical physics.

4.3 Intermittent anticipatory bias control

Lakie et al [35] studied a balancing of a physical inverted pendulum that was manually controlled by a human hand through a compliant linkage of a spring, where the spring and the motions of the human hand corresponded, respectively, to a compliant tendon and active contractions of muscles. Dynamics of the pendulum in their study might be quite similar to those in the subjects of one-sided intermittent type in our study. Although mechanisms for stabilization in both balancing tasks are likely the same. Lakie et al and related studies [34], [36] suggested that an anticipatory and impulsive control of the bias for the compliant spring was responsible for the stabilization, based on the fact that the reaction times (time-lags in cross-correlations) between motions of the pendulum and the human motion were close to zero in their experiments. In other words, the model of anticipatory bias control might consider a state predictor for compensating the delay-induced instability. It should be mentioned that the intermittent control that utilizes the stable manifold during the off-period determines the active feedback torque based on the delay-affected state of the system, and thus it does not necessarily require a state predictor for robust stabilization. Note, however, that a combination use of the stable manifold during the off-period and an internal model during the on-period is effective as modeled by [31].

Note that changes in the bias (the length of contractile muscles) in the model of bias control might be regarded as a sort of virtual equilibrium-point trajectory [65] that has been considered during human arm reaching movement. However, unlike in the reaching movement, the changes are impulsive in the bias control model during quiet standing. It would be interesting issue to examine whether or not release-timings of the active contractions in the experiment of Lakie et al [35] are located close to the stable manifold during the off-period.

4.4 Act-and-wait control: Is intermittency necessary for stabilization?

A main point of discussion here is that stability and intermittent dynamics may or may not be separate issues. In other word, a question is whether or not the intermittency (the off-period) is necessary for stabilization of the inverted pendulum with delay feedback control. As we discussed above, appropriate tuning of the feedback gain parameters in the continuous PD and PDA controllers do not require the off-periods for stabilization [42], [43], [60], [44] even with no help of a state predictor. Of course, the presence of a sensory dead zone in those stable control systems can lead to intermittent dynamics, but it has no effect on stability. In other words, the intermittency may simply be an epiphenomenon or perhaps be beneficial in other ways, e.g. minimize energy requirements as we demonstrated in this paper.

However, there are versions of the intermittent control where the presence of the off-period is critical. The “act-and-wait control” is such an important version of the intermittent control [45], [46], [47]. This control requires a (periodic) sequence of off-periods, and each off-period should be larger than the delay time for stabilizing a delayed feedback control system. This is because, intuitively speaking, destabilizing effect of delay (a memory sequence of states) is forgotten during the off-period, which reduces a problem of stabilization of infinite dimensional delay differential equation into a problem of allocation of finite number of poles. Thus, the intermittent appearance of off-period is critical for the act-and-wait control, as for the intermittent control that utilizes the stable manifold during the off-period.

However, we should note that there is an essential difference between the act-and-wait-control and the intermittent control that utilizes the stable manifold during the off-period. That is, converging dynamics during the on-period of active feedback control is dominantly responsible for the stabilization in the act-and-wait control, whereas transient converging but unstable dynamics during the off-period is important in the intermittent control that utilizes the stable manifold. Stabilization by dynamics during the on-period is also the case for the aforementioned models of intermittent dynamics generated by a sensory dead zone [42], [43]. In this sense, the intermittent control that utilizes the stable manifold is a sort of singularity among various versions of the intermittent control. The present study demonstrated successfully that the stabilization strategy adopted by the majority of subjects was better characterized by and consistent with the specific intermittent control of this sort.

As a computationally positive aspect of the intermittency, Gawthrop et al. [38] have proposed that the intermittency is associated with a computational time necessary for the central nervous system to perform state estimation and prediction. Along this line, Loram et al [39] and van de Kamp et al [40] showed that the reaction time, possibly representing the refractory period in a visuomotor tracking task altered depending on the complexity (the order) of controlled mechanical loads, and claimed that the result could be interpreted by the computational time that induces the behavioral intermittency. However, the results of this study did not necessarily support the existence of delay-compensation that might be achieved by a state estimator and predictor in the central nervous system. Instead, the present study suggests that the subjects in the intermittent types adopted the delay-affected-state-dependent timing strategy that takes advantage of the stable dynamics of the unstable load, which contributes to stabilization of the unstable load.

4.5 Energetic aspects

Similar to the theoretical model simulations with the intermittent control [31], [32], the subjects who adopted the intermittent control strategy could achieve flexible upright stabilization, in which no active effort was made to get the state of the pendulum close to the upright state, but the stable dynamics of the unstable saddle point brought the state of the pendulum to the upright state along the stable manifold without any active effort. The role played by the short-lived active torques was just to move the state of the pendulum close to the stable manifold, not to the upright state itself, which could be achieved with less effort and more robustness than forcing the state of the pendulum directly to the upright state. This is why the intermittent strategy was more energetically efficient than the continuous control. Because of the energetic efficiency of the intermittent strategy, the subjects of continuous control type have the potential to change their strategy from the continuous impedance control to the intermittent control, if they continue the trials. Indeed, Franklin et al [13] suggested that, for human arm reaching movement with an unstable force field, feedback gains were up-regulated with increased uncertainty in the knowledge of the dynamics to counteract any errors or disturbances and ensure accurate and skillful movements.

Of course, the total energy requirement is not only the mechanical energy but the amount of electro-chemical energy that must be consumed by the neurons in the central nervous system in order to compute whatever strategy that is being used. Although we showed that the intermittent strategy was more energetically efficient than the continuous impedance-like control from the point of view of mechanical energy, the computational complexity and thus the corresponding energy consumption might be much higher in the intermittent strategy. Thus, the present study should be considered as a basis of further argument that the intermittent-type control is better energetically than the continuous control.

4.6 Power-law behavior

Sway of the pendulum controlled by the subjects of intermittent types exhibited the double power-law behavior in the low frequency band (0.1–.7 Hz) that characterizes human postural fluctuation [52], in contrast to the white-noise-like non-scaled flat PSD shape in the subjects of continuous type. The intermittent control model proposed by Bottaro et al [31] and Asai et al [32] can exhibit human-like postural sway with the power-law behavior when it is weakly perturbed by periodic and/or random forcing mimicking the endogenous hemodynamic perturbation [53], [66], suggesting that fluctuation of upright posture is established by the intrinsic oscillatory dynamics of intermittent control, rather than driven by noise.

As discussed by previous studies [32], [43], [53], the double power-law behavior is one of the hallmarks indicating that intermittency is involved in the underlying dynamics. Thus, the power-law behavior observed in the subjects of intermittent types also supported that those subjects adopted the intermittent control strategy.

4.7 Remarks and remaining issues

It is important to mention that there are limitations to using a computerized virtual model for postural control. For example, the dynamics for the virtual task are 2-D whereas in the ‘real world’ the dynamics occur in 3-D. Also the computerized virtual model used in this study relies only on visual feedback, whereas the fact that normal subjects can maintain balance with eyes closed, indicating that other forms of sensory information are also important for stabilizing the upright posture.

There are several issues that should be addressed in our future work. As the first issue, one can argue that the variability in outcomes shown in Fig. 2 can be simply due to the fact that the strategies exhibited by the subjects are still in transient learning state, and they are not fixed characteristic of the subjects. In other words, one might ask whether a subject who utilizes the continuous strategy, for example, is always the continuous type strategy user, even when tested the next day or after a few days of practice.

As the second issue, there is not a very good relationship between EMG activity and the force generated by a muscle [67]. Typically, there are electro-mechanical delays between EMG onset and force onset of 60–0 ms, and then between EMG offset and force offset of 120–50 ms. These delays change in complex ways depending on the type of movement that subject is undergoing. Thus, the onset and offset times of EMG in our experimental setup are not necessarily the times when muscle forces were generated.

4.8 Conclusion

Most of the previous studies that examined stabilization of unstable dynamics through motor learning assume continuous-time control. Although there might be a considerable difference in the goal of motor task between voluntary arm movements and postural balancing [22], considering a possibility that the central nervous system generates discontinuous motor commands intermittently provides novel opportunities to better understand motor control and dysfunctions. To our best knowledge, the present study first demonstrates a psychophysical experiment for motor learning during persistent movement, which optimizes the timing-strategy by intermittent control.