Effects of Error Modulation-Based Visual and Haptic Feedback Fusion Strategies on Motor Learning and Motivation

Meng, Jingyan; Li, Na; Hu, Yang; Zhao, Dazheng; Li, Guoning; Hu, Jingyan; Song, Tao; Ma, Yehao; Fu, Rongzhen; Zuo, Guokun; Tao, Liang; Tang, Min; Liu, Yunfeng; Shi, Changcheng

doi:https://doi.org/10.1155/2023/8974148

Applied Bionics and Biomechanics

On this page

Abstract Introduction Methods Results Discussion Conclusion Abbreviations Data Availability Ethical Approval Conflicts of Interest Authors’ Contributions Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2023 | Article ID 8974148 | https://doi.org/10.1155/2023/8974148

Effects of Error Modulation-Based Visual and Haptic Feedback Fusion Strategies on Motor Learning and Motivation

Jingyan Meng,^1,2,3Na Li,^2,3Yang Hu,^1,2,3Dazheng Zhao,^1,2,3Guoning Li,^2,3,4Jingyan Hu,^2,3Tao Song,^2,3Yehao Ma,⁵Rongzhen Fu,^2,3Guokun Zuo,^2,3,4,6Liang Tao,⁷and Min Tang⁷ et al.

Academic Editor: Cristiano De Marchis

Received03 Nov 2022

Revised20 Mar 2023

Accepted14 Jun 2023

Published19 Jul 2023

Abstract

Visual or haptic feedback based on error modulation has been used to improve the effect of robot-assisted rehabilitation training. However, there are several investigations on the effects of error modulation-based visual and haptic feedback fusion strategies on motor learning and motivation. To observe the influence of different feedback fusion strategies on motor learning and motivation, a parallel controlled study was conducted, dividing 30 healthy subjects into three groups with similar skill levels. The no error modulation group received visual and haptic feedback without error modulation; the visual amplification haptic reduction group received visual error amplification combined with haptic error reduction, and the visual reduction haptic amplification (VRHA) group received visual error reduction combined with haptic error amplification. Each subject implemented a trajectory-tracking task with an upper limb rehabilitation robot. They went through baseline, training, assessment, and generalization tests and completed 340 consecutive tracking movements. To evaluate motor learning and motivation, the average tracking error, the root mean square (RMS) of surface electromyography (sEMG) signals, and the intrinsic motivation inventory scale were all examined. In the assessment tests, the average tracking error was significantly decreased in all three groups. In particular, the VRHA group had a larger reduction in average tracking error in the generalization test, lower RMS of sEMG signals both in the assessment and generalization tests and higher perceived competence in the assessment tests. The VRHA fusion strategy significantly improved the subjects’ motor learning and transfer ability, decreased muscle activation, and increased motor learning motivation. These findings may provide some new insights for multisensory feedback fusion technology in the application of rehabilitation robots.

1. Introduction

It is reported that about 75% of stroke patients have upper limb motor impairment, which is caused by brain damage and severely affects their daily life [1]. Improving upper limb motor function has been clinically recognized as a research priority [2]. Conventional rehabilitation is commonly treated with medicines and simple manual or instrumental therapy. However, the tedious nature of manual or instrumental rehabilitation leads to low patient compliance and initiative, which harms the effectiveness of rehabilitation training. In recent years, end-effector upper limb rehabilitation robots (EULRR) have been widely applied in the rehabilitation process of stroke patients [2]. Virtual reality technology is primarily applied to provide patients with context-rich interactive training tasks and audio–visual feedback. Additionally, human–robot interaction technology is made to provide patients with the appropriate haptic feedback. Through repetitive, goal-oriented practice and multisensory feedback, the rehabilitation robotic system improves patients’ motor learning abilities [3]. A major technical challenge that needs to be solved is how to enhance the training effects of rehabilitation robots through the optimal integration of multisensory feedback [4].

Visual or haptic feedback with error amplification or error reduction has been extensively explored and considered a promising human–robot interaction-based rehabilitation technology. A systematic review found that haptic error amplification was overall more effective than conventional repetitive practice and haptic error reduction on motor performance [5]. Shum et al. [6] found that visual error amplification improved bilateral upper limb recovery in hemiplegic cerebral palsy patients. However, most of the existing studies focus only on error-based single-sensory feedback. They discovered that visual error amplification had the potential to trigger motor recovery by awakening the damaged nervous system [7], visual error reduction was more beneficial for older with limited cognitive resources to enhance motor control [8], haptic error amplification had more potential to facilitate motor learning in continuous and discrete tasks [9], and more severe patients were suitable for haptic error reduction with higher stiffness gains [10]. Substantial evidence suggested that multisensory feedback fusion might further improve motor learning in stroke patients. The visual and haptic error amplification feedback fusion group showed faster motor adaptation than the visual error amplification group, higher attention levels, and satisfaction [11]. Visual and haptic error amplification feedback fusion promoted upper limb motor recovery in stroke patients [12, 13]. The benefits of visual error amplification combined with haptic error amplification have been studied, but there is a lack of study on the effects of visual error reduction combined with haptic error amplification on motor learning.

Previous studies have also shown that visual or haptic feedback with error amplification or error reduction has different effects on motivation. Visual error reduction increased patients’ self-confidence since they were highly sensitive to failure [14], while visual error amplification was specially challenging [15]. Haptic error reduction improved perceived competence and satisfaction [16], while haptic error amplification decreased perceived competence and enjoyment [9]. Overall, visual or haptic error reduction feedback has positive effects on motivation, while visual or haptic error amplification feedback may have negative effects on motivation. We want to know if the combination of visual error amplification and haptic error reduction or visual error reduction and haptic error amplification has positive effects on motivation. However, there is a lack of studies on the impact of error amplification and error reduction feedback fusion with different sensory channels on training motivation.

In this study, we designed a trajectory-tracking task to compare the effects of three feedback fusion strategies on motor learning and training motivation. The no error modulation (NEM) group received visual and haptic feedback without error modulation, the visual amplification haptic reduction (VAHR) group received visual error amplification combined with haptic error reduction, the visual reduction haptic amplification (VRHA) group received visual error reduction combined with haptic error amplification. Based on previous studies, we hypothesized that subjects in VAHR and VRHA groups would have better performance on motor learning and motivation than the NEM group. This study aims to provide some new insights for multisensory feedback fusion technology in the application of rehabilitation robots.

2. Methods

2.1. EULRR and Surface Electromyography (sEMG) Acquisition System

EULRR is a five degree-of-freedom rehabilitation robot device for upper limb rehabilitation designed by the Motor Rehabilitation and Medical Robotics Team at the Ningbo Institute of Materials Technology & Engineering, Chinese Academy of Sciences [17]. Figure 1 shows the components of the system; the rehabilitation robot body consists of three servo motors. According to the characteristics of the trajectory-tracking task, the Z-axis of EULRR was not utilized in this experiment; subjects only manipulated the rocker in the X and Y planes. TRIGNO wireless sEMG system (Delsys Inc., Massachusetts, USA) was used to record the sEMG signals [18], and the sampling frequency was set at 2,000 Hz. The four upper limb muscles involved in the trajectory-tracking task were selected: anterior deltoid (AD), biceps brachii (BB), flexor carpi radial (FCR), and extensor carpi radial (ECR). Before the start of the experiment, the four muscles of the subject’s right arm were cleaned with scrub cream and alcohol to reduce the resistance between the electrodes and the skin, and the sEMG sensors were attached to the four muscles parallel to the muscle fiber direction, as detailed in Figure 1. After the end of the experiment, we preprocessed the collected sEMG signals offline. We used a Butterworth bandpass filter at 50–400 Hz to obtain the useful frequency band of the sEMG signals. To eliminate direct current offsets, we subtracted the average value of the sEMG signals. Finally, we applied a full-wave rectification and a low-pass filter at 5 Hz to obtain the linear envelope of the sEMG signals.

2.2. A Trajectory-Tracking Task and the Generalization Experimental Protocols

The virtual environment was designed using Unity 2019.1.0a8 (Unity Technologies, California, USA): a trajectory-tracking task. The task was displayed in two dimensions on the computer screen, and the position of the rocker was mapped to an agent point in the virtual environment. After the task started, the target point moved on the trajectory at a special rate. The subject’s goal is to move the rocker to catch up with the target point to get a high score as possible, and performance-based concurrent visual feedback was provided during the execution of the task. When the target point arrived at the starting point, it was regarded as the end of a consecutive tracking movement. At this moment, the audio would emit auditory feedback of “ding,” and the score as results-based visual feedback would be displayed, see Figure 1 for details. Equation (1) describes the tracking error D_i between the actual point () and the target point (). Equation (2) describes the method for calculating the average tracking error, and N is the collection sample size in a trajectory-tracking movement. The score is calculated based on the average tracking error D_average and the maximum error D_max [19], see Equation (3) for details. D_max was set to 0.04 m in this experiment, which was obtained by repeated measurements. When D_average is greater than D_max, the subject is considered to have a low ability to perform the task, at which point the score is zero. The score was provided only as results-based visual feedback at the end of each consecutive movement and was not used for assessment.

The generalization experiment was designed to evaluate motor learning transfer from one task to another new, similar task. In the generalization experiment, the trajectory was rotated by 90°, and the working range changed from right-to-left to top-to-bottom. All other parameters remained the same as the trajectory-tracking task, such as the speed of the target point and guidelines for calculating scores.

2.3. Error Modulation-Based Visual and Haptic Feedback Strategies

Visual feedback without error modulation: provided true visual feedback of the rocker position in the virtual environment. The position of the agent point was the actual position of the rocker (Figure 2).

2.3.1. Visual Error Reduction [8]

Visually reduced the error between the target point and the actual point. Equation (4) describes the position of agent point under visual error reduction modulation (). The position of the agent point was the position where the actual tracking error was reduced with a proportional gain (Figure 2). Based on a previous study [8], the visual error reduction parameter was 0.5 in this experiment.

2.3.2. Visual Error Amplification [7]

Visually magnified the error between the target point and the actual point. Equation (5) describes the position of the agent point under visual error amplification modulation (). The position of the agent point was the position where the actual tracking error was increased with a proportional gain k₂ (Figure 2). According to a previous study [13], the visual error amplification parameter was 1.5 in this experiment.

Haptic feedback without error modulation: EULRR only provided friction and gravity compensation; it did not provide assistance or resistance force. Subjects can move freely in the X and Y planes.

2.3.3. Haptic Error Reduction [9]

In addition to providing friction and gravity compensation, an assistant force F_ER was provided to help the subject reach the target point. Equation (6) describes the tracking error of the actual point relative to the target point. Equation (7) describes the change of F_ER with e under the modulation of haptic error reduction gain . F_ER proportionally decreased with e decreased (Figure 3). Through repeated tests before the start of the experiment, was in this experiment.

2.3.4. Haptic Error Amplification [9]

In addition to providing friction and gravity compensation, a resistance force F_EA was provided to prevent the subject from reaching the target point. To prevent too large a force to become dangerous [20], haptic error amplification is limited to a certain area. When the tracking error is greater than the maximum error , haptic feedback without error modulation is provided. Equation (8) describes the change of F_EA with under the modulation of actual haptic error amplification gain . F_EA proportionally increased with decreased. Equation (9) describes the relationship between and . gets maximum when is equal to zero, and gets minimum when larger or equal to the maximum error D_max. Through repeated tests before the start of the experiment, haptic error amplification gain was 300 N · m in this experiment.

2.4. Subjects

Thirty healthy, right-handed adults (16 males and 14 females, with average ages of 24.9 ± 2.5 years) without any history of neurological disorders were recruited to participate in this experiment. Subjects were randomly divided into three groups: NEM, VAHR, and VRHA. There were no significant differences in gender, age, and average tracking error among the three groups of subjects after statistical analysis (), detailed information is shown in Table 1. The study protocol was approved by the Ningbo Institute of Materials Technology & Engineering, Chinese Academy of Sciences, and all subjects gave written informed consent.

2.5. Experimental Protocol

A parallel controlled experiment was conducted on separate 3 days, and each subject went through the same test phases, as detailed in Figure 4.

On day 1, subjects started with a Familiarization phase to adapt the EULRR and the trajectory-tracking task. They performed 20 consecutive tracking movements under visual and haptic feedback without error modulation. After a 5-min break, subjects performed the Baseline1, where they played again 20 consecutive tracking movements under the visual and haptic feedback without error modulation. The purpose of the phase was to test the initial skill level of subjects, who would be divided into three groups with nonsignificant differences in average skill level.

On day 2, the subjects’ sEMG signals were recorded once every 20 consecutive tracking movements to understand muscle activation of subjects’ upper limbs during the trajectory-tracking task and observe the differences in physical effort under different feedback strategies. Subjects performed 20 consecutive tracking movements under the visual and haptic feedback without error modulation during Baseline2, and the intrinsic motivation inventory (IMI) scale was completed after Baseline2. After a 5-min break, subjects performed the Baseline generalization, where the trajectory was rotated by 90°. They repeated 20 consecutive tracking tasks under visual and haptic feedback without error modulation. After a 5-min break, subjects completed 180 consecutive tracking movements in NEM, VAHR, or VHAR groups with different feedback strategies during Training. Training1 refers to the initial 60 movements, Training2 to the middle 60 movements, and Training3 to the last 60 movements. There was a 5-min break at the end of every 20 consecutive movements, and IMI questions needed to be completed at the end of every 60 movements. During Short-term assessment and Short-term generalization assessment, subjects underwent the same experimental procedures as the Baseline2 and Baseline generalization.

On day 3, occurred within 1–3 days after day 2, subjects were invited to perform Long-term assessment and Long-term generalization assessment. To evaluate the influence of long-term retention on motor learning, these phases utilized the same experimental procedures as the Baseline2 and Baseline generalization.

After Baseline2, Training1, Training2, Training3, Short-term assessment, and Long-term assessment, subjects were asked to complete the IMI scale. IMI questionnaire has been widely used to assess motor learning motivation [9, 15, 16, 21]. The full questionnaire consists of seven motivational subscales [22], three of which were selected in this experiment: interest/enjoyment, perceived competence, and effort/importance, as detailed in Table 2. Subjects chose a Likert scale between 1 and 7 points according to their subjective feelings, with 1 indicating “I disagree completely” and 7 indicating “I agree completely.” The subjects’ previously filled-out contents can be seen.

2.6. Assessments of Motor Learning and Training Motivation

All subjects’ kinematic information, the sEMG signals, and IMI questionnaire results were collected.

Average tracking error was used to assess the accuracy of the movement. The change in the average tracking error from Baseline1 to Short-term assessment or Long-term assessment for evaluating the short-term or long-term effects of motor learning. The change in the average tracking error from Baseline generalization to Short-term generalization assessment or Long-term generalization assessment was used to evaluate the short-term or long-term effects of motor learning transfer. In addition, exponential curves were fitted to evaluate the motor learning effects of different feedback strategies during the Training period [7]. As in Equation (10), y is the average tracking error, and x is the number of groups during Training (every 20 consecutive tracking movements as a group). Based on this, a is the amount of learning (the differences of the average tracking error between the beginning and the end of training), b is the rate of learning (the number of groups corresponding to when 63% amount of learning is reached), c is the value of the average tracking error convergence at the end of Training.

The root mean square (RMS) of the sEMG signals was used to evaluate the effects of different feedback strategies on muscle activation. After obtaining the envelope of the collected sEMG signals, we calculated the RMS value of the sEMG signals containing 20 consecutive tracking movements offline, as shown in Equation (11), E_i is the ith sEMG signals, and N is the collection sample size.

The IMI scale was used to evaluate motor learning motivation. Subjects’ responses to interest/interest, perceived competence, and effort/importance were collected, with Baseline2 used as a reference to compare the amount of change in responses in other periods.

2.7. Statistical Analysis

To evaluate motor learning effects, we analyzed the changes in average tracking error between Baseline1 and Training1, Short-term assessment (last 10 consecutive tracking movements), and Long-term assessment (first 10 consecutive tracking movements) using independent t-tests within a group. To evaluate motor learning transfer, we analyzed the changes in average tracking error between Baseline generalization and Short-term generalization assessment and Long-term generalization assessment using independent t-tests within a group. A one-way analysis of variance (ANOVA) was used to analyze the average tracking error between groups during Baseline1, Training, and assessment, the reduction in average tracking error between groups during generalization, and the RMS of sEMG signals between groups during all periods. If the ANOVA was significant, multiple comparisons were carried out using the least significant differences method. We compared the changes in responses at each phase using the nonparametric Kruskal–Wallis one-way ANOVA with training strategies as the main factor. The differences were considered statistically significant at . Data processing and statistical analysis were performed in Microsoft Office Excel 2013 (Microsoft, Washington, USA), and measurement information was presented as .

3. Results

3.1. Effect of Training Strategies on Motor Learning

The exponential curve fits for the different feedback strategies during Training can be seen in Figure 5, which shows that subjects in the VRHA group had the biggest a value and smallest b value, and subjects in the VAHR group had the smallest c value.

(a)

(b)

(c)

Table 3 shows the average tracking error for the three groups of subjects at Baseline1, Training1, and assessment. During the Training1 period, the average tracking error significantly decreased in NEM and VAHR groups () while significantly increased in VRHA group compared to Baseline1 (). In the Short-term and Long-term assessments, the average tracking error was significantly decreased in all three groups (). Figure 6 compares the differences of average tracking error between groups. There were no significant differences between groups at Baseline1 (). During Training, there were significant differences between groups (); subjects in VRHA group had the biggest average tracking error, while subjects in VAHR group had the smallest. During both Short-term and Long-term assessments, there was a smaller average tracking error in VRHA group, but the error reduction was nonsignificant between groups ().

3.2. Effect of Training Strategies on Motor Learning Transfer

Table 4 shows the average tracking error for the three groups of subjects during the generalization phases. In the Short-term generalization assessment, only the VRHA group significantly decreased the average tracking error compared to the Baseline generalization (). In the Long-term generalization assessment, all three groups of subjects significantly decreased the average tracking error compared to the Baseline generalization (). Figure 7 shows the reduction in the average tracking error in Short-term and Long-term generalization assessment. The VRHA group significantly reduced the average tracking error than the NEM group (), and the VAHR group also reduced the average tracking error more than the NEM group, although the reduction did not reach significance ().

3.3. Effect of Training Strategies on Muscle Activation

The RMS values of four muscles’ sEMG for a typical subject (subject no. 5) from the NEM group are shown in Figure 8(a). Figure 8(b) shows the RMS values of one muscle’s sEMG for three typical subjects (subjects nos. 5, 14, 22) from NEM, VAHR, and VRHA groups. There were no significant differences between groups in the RMS of sEMG signals at Baseline2 and Baseline generalization (). Figure 9 shows the RMS of sEMG signals in the Training, Short-term assessment, and Short-term generalization assessment. In general, subjects in the VRHA group had a lower RMS than those in the NEM group. During Training, the AD of subjects in the VRHA group had significantly lower RMS than the NEM group (). In the Short-term assessment, the AD of subjects in the VRHA group still had significantly lower RMS than the NEM group (). In the Short-term generalization assessment, subjects in the VRHA group showed significantly lower RMS of BB and AD than the NEM group (), and significantly lower RMS of AD than the VAHR group ().

(a)

(b)

Figure 8

RMS values of sEMG for subjects during 20 consecutive movements in the Training process. (a) RMS values of four muscles’ sEMG for a typical subject (subject no. 5) from the NEM group. (b) RMS values of one muscle’s sEMG for three typical subjects (subject nos. 5, 14, 22) from NEM, VAHR, and VRHA groups. BB, biceps brachii; AD, anterior deltoid; FCR, flexor carpi radialis; ECR, extensor carpi radialis; NEM, visual and haptic feedback without error modulation; VAHR, visual error amplification combined with haptic error reduction; VRHA, visual error reduction combined with haptic error amplification.

(a)

(b)

(c)

Figure 9

The RMS of sEMG signals in the Training, Short-term assessment, and Short-term generalization assessment. (a) Training, (b) Short-term assessment, and (c) Short-term generalization assessment. NEM, visual and haptic feedback without error modulation; VAHR, visual error amplification combined with haptic error reduction; VRHA, visual error reduction combined with haptic error amplification; RMS, root mean square; BB, biceps brachii; FCR, flexor carpi radialis; ECR, extensor carpi radialis; AD, anterior deltoid. ; .

3.4. Effect of Training Strategies on Motivation

Figure 10 shows the changes in responses to the IMI questionnaire after Baseline2, Training, and assessment. In the Short-term and Long-term assessments, subjects in the VRHA group significantly increased perceived competence (). There were no significant differences in interest/enjoyment and effort/importance between groups in Baseline2, Training, and assessment (). In particular, subjects in the NEM group significantly improved perceived competence only during Training1 (), subjects in the VAHR group significantly improved perceived competence during Training (), and subjects in the VRHA group significantly improved perceived competence in all periods (). Only subjects in the VAHR group significantly improved effort/importance during Training1 ().

(a)

(b)

(c)

4. Discussion

Visual or haptic feedback based on error modulation has been extensively studied to improve the effectiveness of robot-assisted rehabilitation training. However, there are several investigations on the effects of error amplification and error reduction feedback fusion with different sensory channels on motor learning and motivation. In this study, we designed a trajectory-tracking task to evaluate the effects of three error modulation feedback fusion strategies (i.e., NEM, VAHR, VRHA) on motor learning and motivation.

During the Training period, subjects in the VAHR group had the best motor performance, and the VRHA group had the worst motor performance. This is in line with previous studies on haptic error modulation [9, 23], which discovered that haptic error reduction enhanced subjects’ performance while haptic error amplification degraded it. Based on the trajectory-tracking task, visual error amplification enhanced performance by repeatedly correcting for errors [7], while visual error reduction degraded performance may be due to incorrect delivery of task-relevant information [24]. Therefore, visual error amplification combined with haptic error reduction feedback fusion further enhanced performance, and visual error reduction combined with haptic error amplification feedback fusion further degraded performance. However, the relatively high motor performance of subjects in the VAHR group may limit their learning potential at the beginning of Training. In contrast, the relatively poor motor performance of subjects in the VRHA group may lead to more improvement at the beginning of Training. Therefore, the VRHA group had the largest amount of motor learning and the fastest learning rate during Training, which was indicated by the results of the exponential fit curves in Figure 5.

During the Short-term and Long-term assessment periods, the subjects in all three groups had significant improvement comparing with Baseline1, indicating that the motor learning did happen in our study. The subjects in the VRHA group had a relatively better motor performance with the lowest average tracking error, while there were no statistical differences between groups. This may be explained by the specificity-of-learning hypothesis [25], which states that motor learning is most effective when subjects receive the most important sensory information needed to complete the task during the Training period. In our experiments, subjects may mainly rely on performance-based concurrent visual feedback to catch up with target points during the trajectory-tracking task. Therefore, all three groups significantly improved performance in the assessment period may be due to the real-time visual feedback play a major role in the trajectory-tracking task.

Generalization is an important aspect of motor learning to evaluate transfer. The subjects in all three groups had significant improvement comparing with Baseline generalization in the Long-term generalization assessment. Only subjects in the VRHA group improved transfer ability in the Short-term generalization assessment. Both in the Short-term and Long-term generalization assessments, the VRHA group had significantly higher transfer ability than the NEM group. The trajectory-tracking task requires both temporal and spatial alignment with the target point. It has been demonstrated that haptic feedback is as effective as visual feedback when learning spatiotemporal aspects of a motor task, and haptic feedback does not further facilitate motor learning when visual feedback provides sufficient information [24]. Furthermore, a related study [15] also found that haptic error amplification enhanced the transfer of the practiced asymmetric gait pattern to free walking. It is possible that subjects in the VAHR and NEM groups received sufficient information provided by visual feedback, causing haptic feedback was redundant and helpless for task learning. However, when subjects in the VRHA group found that performance-based feedback was inconsistent with results-based feedback, they may rely on haptic error amplification to learn the trajectory-tracking task. This task-related information transmitted by the haptic error amplification may cause subjects in the VRHA group significantly improve transfer ability.

The results of sEMG showed that visual feedback may have a significant effect on motor control. A previous study [26] has shown that haptic error amplification leads to higher muscle activation, and haptic error reduction leads to lower muscle activation. However, this study showed that subjects trained under visual error reduction combined with haptic error amplification had lower muscle activation. The subjects may subjectively decrease physical effort due to good performance-based visual feedback provided by visual error reduction [27]. On the other hand, they may reduce unnecessary muscle use due to relying more on haptic error amplification to learn to perform the task. A study [24] has shown that experts generally achieve maximum accuracy with minimum effort, so visual error reduction combined with haptic error amplification may stimulate subjects’ potential and help them quickly reach the expert level.

Motivation is a key factor for engaging trainees in extensive, repetitive training tasks. Previous studies [9, 16] have reported haptic error amplification has negative effects on motivation. However, the results of the present study showed that visual error reduction combined with haptic error amplification feedback fusion improved perceived competence. A large score increment in the VRHA group may enhance subjects’ satisfaction and self-efficacy. On the other hand, it is also possible that the perceived competence brought by visual error reduction overshadows the frustration brought by haptic error amplification. Therefore, maintaining subjects under the condition with increasing scores can lead to higher perceived competence, which may indicate new ideas for designing motivation-enhancing rehabilitation training strategies.

Modern rehabilitation theory views neurorehabilitation as a motor relearning process, and motor learning promotes the reorganization of brain neural circuits to encode new learning experiences and enable behavioral change [28]. Therefore, motor learning is related to stroke recovery and neurorehabilitation. Providing multisensory feedback during motor learning may help correct movement, strengthen the relationship between external stimulus and internal responses, and increase self-efficacy and motivation. Our findings may have important implications for optimizing multisensory feedback and improving the effectiveness of robot-assisted rehabilitation. Visual error reduction combined with haptic error amplification could accelerate the motor learning process, enhance motor skill transfer, decrease muscle activation, and increase motor learning motivation. The results of motor learning and motivation found in this study may be applied in the robot-assisted rehabilitation training of stroke patients; the task difficulty can be reduced by modifying trajectory or decreasing the haptic error amplification feedback parameter to match the motor ability of stroke patients.

There are still several limitations in this study. First, a previous study [9] showed that the effectiveness of the training strategy depended on motor task characteristics. This study was conducted only for a trajectory-tracking task; a variety of task types should be considered in the future. Second, according to challenge point theory [29], optimal learning is formed when the difficulties of the task match the individual’s skill level, and it has also been shown that there are optimal error modulation parameters to maximize the effectiveness of the training strategy [30]. The present study used fixed error modulation parameters, and the difficulty of the task remained constant as the skill level increased, which was detrimental to subjects’ motor learning in the later phase. Finally, the sample size was small, and the subjects were all healthy. Although motor learning may share the same brain mechanisms as neurorehabilitation [24], stroke patients have a different learning status and motivation during the rehabilitation. The investigations on error modulation-based visual and haptic feedback fusion for rehabilitation patients are needed in the next study.

5. Conclusion

In summary, we designed a trajectory-tracking task to evaluate the effects of three error modulation feedback fusion strategies (i.e., NEM, VAHR, VRHA) on motor learning and motivation. The results showed that all three groups significantly improved motor learning ability, particularly subjects in the VRHA group had higher transfer ability and perceived competence and reduced unnecessary muscle activation. Our study provided a new research idea to optimize the multisensory feedback technology for rehabilitation robots. Providing visual error reduction and haptic error amplification feedback strategies during rehabilitation training may not only improve motor learning efficiency and active participation motivation but also have a potential for facilitate a higher dose of rehabilitation training. Adaptive optimization of visual and haptic feedback fusion strategies based on personalized rehabilitation requirements of patients may accelerate the recovery of upper limb motor function; this will be studied in the future.

Abbreviations

NEM:	No error modulation
VAHR:	Visual amplification haptic reduction
VRHA:	Visual reduction haptic amplification
EULRR:	End-effector upper limb rehabilitation robot
sEMG:	Surface electromyography
AD:	Anterior deltoid
BB:	Biceps brachii
FCR:	Flexor carpi radialis
ECR:	Extensor carpi radialis
IMI:	Intrinsic motivation inventory
RMS:	Root mean square.

Data Availability

Data supporting this research article are available from the corresponding author or first author upon reasonable request.

Ethical Approval

This research was conducted by the Declaration of Helsinki (1964) and approved by the Ningbo Institute of Materials Technology & Engineering, Chinese Academy of Sciences.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

JM contributed to the analysis, data collection, and summarization. NL helped to build the experiment platform. YH, GL, TS, and RF solved the problems of rehabilitation robot control. DZ and YM contributed to the acquisition and processing of surface EMG signals. JH provided help in virtual environment design. LT and MT provided recommendations on clinical needs and trajectory-tracking task design. GZ, YL, and CS participated in experimental design, writing instruction, and manuscript revision. All authors approved the final manuscript.

Acknowledgments

This research was supported by the National Key R&D Program of China (2022YFC3601701), Ningbo Municipal Science and Technology Innovation 2025 Major Project (2018B10073, 2020Z082, 2020Z022), Director Fund of Ningbo Institute of Materials, Chinese Academy of Sciences (2021SZKY0205), and Zhejiang Basic Public Welfare Research Program (LGF21E050004, LGF21H170002).

References

R. Iandolo, F. Marini, M. Semprini et al., “Perspectives and challenges in robotic neurorehabilitation,” Applied Sciences, vol. 9, no. 15, Article ID 3183, 2019.
View at: Publisher Site | Google Scholar
H. Rodgers, H. Bosomworth, H. I. Krebs et al., “Robot assisted training for the upper limb after stroke (RATULS): a multicentre randomised controlled trial,” The Lancet, vol. 394, no. 10192, pp. 51–62, 2019.
View at: Publisher Site | Google Scholar
M. Maier, B. R. Ballester, and P. F. M. J. Verschure, “Principles of neurorehabilitation after stroke based on motor learning and brain plasticity mechanisms,” Frontiers in Systems Neuroscience, vol. 13, p. 74, 2019.
View at: Publisher Site | Google Scholar
R. Sigrist, G. Rauter, R. Riener, and P. Wolf, “Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review,” Psychonomic Bulletin & Review, vol. 20, pp. 21–53, 2013.
View at: Publisher Site | Google Scholar
L. Y. Liu, Y. Li, and A. Lamontagne, “The effects of error-augmentation versus error-reduction paradigms in robotic therapy to enhance upper extremity performance and recovery post-stroke: a systematic review,” Journal of NeuroEngineering and Rehabilitation, vol. 15, Article ID 65, 2018.
View at: Publisher Site | Google Scholar
L. C. Shum, B. A. Valdés, N. J. Hodges, and H. F. M. Van der Loos, “Error augmentation in immersive virtual reality for bimanual upper-limb rehabilitation in individuals with and without hemiplegic cerebral palsy,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 28, no. 2, pp. 541–549, 2020.
View at: Publisher Site | Google Scholar
Y. Wei, P. Bajaj, R. Scheidt, and J. Patton, “Visual error augmentation for enhancing motor learning and rehabilitative relearning,” in IEEE 9th International Conference on Rehabilitation Robotics, 2005, pp. 505–510, IEEE, Chicago, IL, 2005.
View at: Publisher Site | Google Scholar
I.-S. Hwang, C.-L. Hu, W.-M. Huang, Y.-Y. Tsai, and Y.-C. Chen, “Potential motor benefits of visual feedback of error reduction for older adults,” Journal of Aging and Physical Activity, vol. 28, no. 6, pp. 934–942, 2020.
View at: Publisher Site | Google Scholar
L. Marchal-Crespo, N. Rappo, and R. Riener, “The effectiveness of robotic training depends on motor task characteristics,” Experimental Brain Research, vol. 235, pp. 3799–3816, 2017.
View at: Publisher Site | Google Scholar
J. B. Rowe, V. Chan, M. L. Ingemanson, S. C. Cramer, E. T. Wolbrecht, and D. J. Reinkensmeyer, “Robotic assistance for training finger movement using a Hebbian model: a randomized controlled trial,” Neurorehabilitation and Neural Repair, vol. 31, no. 8, pp. 769–780, 2017.
View at: Publisher Site | Google Scholar
N. Shirzad and H. F. M. Van der Loos, “Error amplification to promote motor learning and motivation in therapy robotics,” in 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 3907–3910, IEEE, San Diego, CA, USA, 2012.
View at: Publisher Site | Google Scholar
F. Abdollahi, M. Corrigan, E. D. C. Lazzaro, R. V. Kenyon, and J. L. Patton, “Error-augmented bimanual therapy for stroke survivors,” NeuroRehabilitation, vol. 43, no. 1, pp. 51–61, 2018.
View at: Publisher Site | Google Scholar
F. Abdollahi, E. D. Case Lazarro, M. Listenberger et al., “Error augmentation enhancing arm recovery in individuals with chronic stroke: a randomized crossover design,” Neurorehabilitation and Neural Repair, vol. 28, no. 2, pp. 120–128, 2014.
View at: Publisher Site | Google Scholar
B. R. Ballester, M. Maier, R. M. San Segundo Mozo, V. Castañeda, A. Duff, and P. F. M. J. Verschure, “Counteracting learned non-use in chronic stroke patients with reinforcement-induced movement therapy,” Journal of Neuroengineering and Rehabilitation, vol. 13, Article ID 74, 2016.
View at: Publisher Site | Google Scholar
L. Marchal-Crespo, P. Tsangaridis, D. Obwegeser, S. Maggioni, and R. Riener, “Haptic error modulation outperforms visual error amplification when learning a modified gait pattern,” Frontiers in Neuroscience, vol. 13, p. 61, 2019.
View at: Publisher Site | Google Scholar
J. E. Duarte and D. J. Reinkensmeyer, “Effects of robotically modulating kinematic variability on motor skill learning and motivation,” Journal of Neurophysiology, vol. 113, no. 7, pp. 2682–2691, 2015.
View at: Publisher Site | Google Scholar
S. Zhang, G. Zuo, C. Shi et al., “The sEMG characteristics of human upper limb during circle drawing on EULRR system,” in 2017 IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), pp. 423–427, Ningbo, China, 2017.
View at: Publisher Site | Google Scholar
C. Wang, S. Zhang, J. Hu, Z. Huang, and C. Shi, “Upper-limb muscle synergy features in human–robot interaction with circle-drawing movements,” Applied Bionics and Biomechanics, vol. 2021, Article ID 8850785, 11 pages, 2021.
View at: Publisher Site | Google Scholar
A. Takai, G. Lisi, T. Noda, T. Teramae, H. Imamizu, and J. Morimoto, “Bayesian estimation of potential performance improvement elicited by robot-guided training,” Frontiers in Neuroscience, vol. 15, Article ID 704402, 2021.
View at: Publisher Site | Google Scholar
S. Rüdt, M. Moos, S. Seppey, R. Riener, and L. Marchal-Crespo, “Towards more efficient robotic gait training: a novel controller to modulate movement errors,” in 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), pp. 876–881, IEEE, Singapore, 2016.
View at: Publisher Site | Google Scholar
D. Novak, A. Nagle, U. Keller, and R. Riener, “Increasing motivation in robot-aided arm rehabilitation with competitive and cooperative gameplay,” Journal of NeuroEngineering and Rehabilitation, vol. 11, Article ID 64, 2014.
View at: Publisher Site | Google Scholar
R. M. Ryan, “Control and information in the intrapersonal sphere: an extension of cognitive evaluation theory,” Journal of Personality and Social Psychology, vol. 43, no. 3, pp. 450–461, 1982.
View at: Publisher Site | Google Scholar
C. K. Williams, L. Tremblay, and H. Carnahan, “It pays to go off-track: practicing with error-augmenting haptic feedback facilitates learning of a curve-tracing task,” Frontiers in Psychology, vol. 7, pp. 1–14, 2016.
View at: Publisher Site | Google Scholar
E. Basalp, P. Wolf, and L. Marchal-Crespo, “Haptic training: which types facilitate (re)learning of which motor task and for whom? Answers by a review,” IEEE Transactions on Haptics, vol. 14, no. 4, pp. 722–739, 2021.
View at: Publisher Site | Google Scholar
L. Proteau, “Visual afferent information dominates other sources of afferent information during mixed practice of a video-aiming task,” Experimental Brain Research, vol. 161, pp. 441–456, 2005.
View at: Publisher Site | Google Scholar
L. Marchal–Crespo, J. Schneider, L. Jaeger, and R. Riener, “Learning a locomotor task: with or without errors?” Journal of NeuroEngineering and Rehabilitation, vol. 11, Article ID 25, 2014.
View at: Publisher Site | Google Scholar
Y.-C. Chen, Y.-T. Lin, G.-C. Chang, and I.-S. Hwang, “Perceptual influences of error size on voluntary force control during a compound sinusoidal force task,” Human Movement Science, vol. 56, no. Part-B, pp. 46–53, 2017.
View at: Publisher Site | Google Scholar
J. A. Kleim and T. A. Jones, “Principles of experience-dependent neural plasticity: implications for rehabilitation after brain damage,” Journal of Speech, Language, and Hearing Research, vol. 51, no. 1, pp. S225–S239, 2008.
View at: Publisher Site | Google Scholar
M. A. Guadagnoli and T. D. Lee, “Challenge point: a framework for conceptualizing the effects of various practice conditions in motor learning,” Journal of Motor Behavior, vol. 36, no. 2, pp. 212–224, 2004.
View at: Publisher Site | Google Scholar
P. N. Parmar and J. L. Patton, “Optimal gain schedules for visuomotor skill training using error-augmented feedback,” in IEEE International Conference on Robotics and Automation (ICRA), pp. 3809–3813, IEEE, Seattle, WA, USA, 2015.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2023 Jingyan Meng et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies