Abstract

The precise capture and identification of movement features are important for numerous scientific endeavors. In this work, we present a novel multimodal sensor, called the resistance/capacitance dual-mode (RCDM) sensor, which effectively differentiates between compression and stretchable strains during tennis motion; meanwhile, it can also accurately identify various joint movements. The proposed wearable device features a seamless design, comprising two separate components: a resistive part and a capacitive part. The resistive and capacitive components operate independently and utilize a resistance–capacitance mechanism to measure pressure and strain signals, respectively. The RCDM sensor demonstrates remarkable sensitivity to strains (GF = 7.84, 0%–140%) and exceptional linear sensitivity (S = 4.08 kPa−1) through capacitance. Utilizing machine learning algorithms, the sensor achieves a recognition rate of 97.21% in identifying various joint movement patterns. This advanced production method makes it feasible to manufacture the sensors on a large scale, offering tremendous potential for various applications, including tennis sports systems.

1. Introduction

The precise monitoring and identification of human body movements are critical for comprehending human behavior and vital signs in various fields, including medical rehabilitation [1], sports training [2, 3], and electronic skin technology [4]. Nevertheless, human body movements include various complex movements, such as elbow bending, four-leg stretching, etc. [5, 6]. Currently, many flexible sensors are in direct contact with the human body and are subjected to both pressure and strain, making it challenging to distinguish between mechanical inputs [7]. Therefore, it is necessary to develop wearable flexible sensors with human motion and posture recognition functions, for example, multimodal sensor [8]. In recent years, various methods have been explored by researchers to develop flexible sensors with multiple modes of operation. A strategy that has been pursued is the development of a unified sensor architecture capable of detecting multiple stimuli concurrently [9, 10]. For example, Song et al. [11] proposed a fabric sensor based on carbon nanotubes, which can realize multidirectional stress monitoring. Qin et al. [12] developed the hydrophobic carbon nanotube (CNT)-filled polyacrylamide hydrogel as a multifunctional skin sensor that displayed high sensitivity to a wide range of strains and linear sensitivity to pressure. Chao et al. [13] have manufactured a skin sensor based on dopamine gel, which can monitor mechanical stress and temperature. However, sensors with a single working mode still face many challenges in practical applications. In addition, the second strategy is to integrate the two sensors into one, and realize the monitoring of complex motion posture by giving play to their respective advantages. Su et al. [14] developed a wearable sensor by incorporating a biocompatible and conductive filler, NaCl-doped agarose gel, into a 3D-printed elastomer mold, which can distinguish between bending and stretching movements. In contrast, Lee et al. [15] proposed a sensor with two anisotropic layers to produce multidirectional strain sensing. This approach allows for capturing strains in both parallel and perpendicular directions, thus addressing the problem of distinguishing between multiple stimuli. However, the stability and mechanical properties of these sensors pose challenges, and the manufacturing process remains complex. Besides, Park et al. [16] proposed a self-driving sensor, which can realize pressure and strain sensing at the same time. While this design offers promising capabilities, the intricate fabrication process increases both the complexity and cost of the sensor [1721]. To address the need for a convenient, wearable, and multimode sensor, further exploration and innovative solutions are required [2225]. At present, flexible wearable sensors have practical application value in various fields such as medical rehabilitation and sports training [20, 26, 27].

Here, we present a novel multimodal sensor, called the resistance/capacitance dual-mode (RCDM) sensor, which effectively differentiates between compression and stretchable strains during tennis motion; meanwhile, it can also accurately identify various joint movements by introducing the recurrent neural network (RNN). From the results, the RCDM sensor features remarkable sensitivity to strains (GF = 7.84, 0%–140%) and exceptional linearity in pressure sensitivity (S = 4.08 kPa−1) through capacitance. The rapid response and recovery times were ∼290 and 300 ms under a stretching strain of 20% and a pressure of 14 kPa. RCDM sensor adopts integrated structure design, which realizes the combination of resistance elements and capacitance elements. The results highlight the capability of the sensor to simultaneously and accurately monitor relative resistance change rate (ΔR/R0) and relative capacitance change rate (ΔC/C0). Also, the casting process for the production of the RCDM sensor provides a scalable and convenient method for large-scale production. With a recognition rate of 97.21% achieved through the use of deep learning algorithms, the RCDM sensor is capable of accurately identifying movement patterns and states of different joints during tennis sports. These results indicate that the RCDM sensor and processing technique have promising potential for various applications, including health monitoring and intelligent sports systems. The findings suggest that the RCDM sensor and its processing technique hold great promise for a range of applications, such as intelligent sports systems.

2. Experiments

2.1. Materials

The polydimethylsiloxane (PDMS) was bought from Dingxu Micro Control Technology Co., Ltd. (Suzhou, China). The polyvinylidene fluoride (PVDF) was purchased from Arkema Fluorochemical Co., Ltd. (Changshu, China). The graphite and kapton were bought from Kaipu Tape Co., Ltd. (Dongguan, China).

2.2. The Preparation of PDMS/PVDF/Graphite Solution

To obtain a uniform and well-dispersed suspension, the dried graphite powder was sonicated in N-dimethylformamide (DMF) at 88% amplitude for 1 hr. The sediment that contained graphite was obtained from the sonicated mixture after it was centrifuged at 9,000 rpm for 15 min. The graphite powder was homogeneously stirred with a PDMS/PVDF solution for 2 hr to generate a PDMS/PVDF/graphite solution at concentrations of up to 12% (w/v).

2.3. The Fabrication of RCDM Sensor

The RCDM sensor is created using a step-by-step casting technique, as illustrated in Figure 1. The process begins by securing a 3D printing mold onto a PDMS substrate. The parts A and B of PDMS are blended in a 10 : 1 ratio, and then poured into the 3D printing mold after eliminating air pockets. The mixture is allowed to cure for 60 min, resulting in a partially cured PDMS substrate, as shown in Figure 1(a). Subsequently, the PDMS/PVDF/graphite mixture is brushed to the semicured PDMS surface, as shown in Figure 1(b). After the PDMS is fully cured, graphite in PDMS/PVDF/graphite will diffuse into the PDMS layer. Then, the lead wire is connected to the conductive copper foil, as shown in Figure 1(c). After PDMS/PVDF/graphite is cured, a layer of PDMS/PVDF with mass ratio of 1 : 10 shall be poured on its surface, as shown in Figures 1(d) and 1(e). When PDMS/PVDF is cured, conductive electrodes are arranged on its side and a layer of PDMS/PVDF/graphite is brushed, as shown in Figures 1(f) and 1(g). As illustrated in Figure 1(h), a layer of PDMS is poured on the semicured surface PDMS/PVDF/graphite as the encapsulation layer.

2.4. Characterization

Figure 1(i) displays the structure configuration of the RCDM sensor device, while Figure 1(j) presents its picture. Figure 1(k) illustrates the scanning electron microscope (SEM) image of PDMS/PVDF layer. Mechanical measurement is carried out with the help of linear mechanical testing machine, while electrical measurement is carried out using a test system consisting of Keithley 4200-SCS and LCR digital bridge TH2826.

3. Results and Discussion

3.1. The RCDM Sensor Sensing Mechanism

To assess the mechanical features, the research team performed both stretching and compression experiments. As depicted in Figure 2(a), the stress–strain graphs display the results of the RCDM sensor and PDMS/PVDF slender during stretching (stretching rate : 10.1 mm/min). According to the results, the fracture RCDM strain sensor can arrive at 460%, and the PDMS/PVDF slender fracture strain can reach at 470%. The compressive performance of the RCDM sensor was evaluated using compression models at a speed of 0.5 mm/min, as depicted in Figure 2(b). An example of the stretching loading–unloading tests at 120% strain is presented in Figure 2(c), while the cyclic compression tests at 17.5% strain are shown in Figure 2(d), revealing a compressive strength of ∼16.8 kPa. These outcomes highlight the dependable and robust mechanical traits of the RCDM sensor, attributed to its seamless design rather than being a combination of two distinct sensing components. As illustrated in Figures 2(e) and 2(f), the direct current–voltage (IV) curves provide an illustration of the electrical behavior of the RCDM sensor when subjected to different stretching and compression loads. The electrical characteristics of the device were investigated by measuring the currents at various voltages ranging from −1 to 1 V. These recorded currents were then utilized to evaluate the performance of the device. The excellent ohmic contact of the device is demonstrated by the linear behavior of the plotted curves, while the reciprocal of the slope of the IV curves can be utilized to determine the resistance of the device. A direct relationship between strain and resistance was suggested as the average resistance values, shown in the insets of Figure 2(e), which increased significantly with stretching levels of 0%, 30%, 60%, 90%, and 120%, with values of 26.4, 36.48, 55.8, 90.36, and 144.72 kΩ, respectively. The inset in Figure 2(f) shows that the resistance of the RCDM sensor was only slightly affected by pressure, with values of 26.4, 26.52, 26.88, and 28.44 kΩ measured at various pressures. Compared to the significant changes in resistance observed under stretching, the minor variation in resistance under pressure offers a more direct means of analyzing the correlation between electrical output and applied load, thereby simplifying the data processing.

The gauge factor, a key performance index of the sensor, is computed as GF = (ΔR/R0). The initial resistance is represented by R0, the change in resistance prior to and subsequent to the application of strain is represented by ΔR, and the strain applied is symbolized by ε. To attain a more comprehensive grasp of the electrical sensing mechanism, the resistance R was partitioned into three components—R1, R2, and R3—that symbolize the peak slope resistance, valley bottom resistance, and peak interior resistance, respectively. A more detailed depiction of the mechanism can be found in Figure 3(a1), which illustrates the electrical model with greater clarity. As shown in Figure 3(a2), to simplify the circuit and streamline the calculation process, the circuit was equivalent based on Thevenin’s theorem, and the detailed resistance can be shown as follows:

In the initial stage (0 < ε < 80%), the valley floor underwent significant stretching that caused numerous conductive pathways to break, resulting in a sudden and virtually infinite increase in R2. In the later stage of strain (0 < ε < 80%), the peak slope slowed down and its interior compressed, leading to a slight decline in R3 and a constant R1. The overall resistance R was a result of the combination of R12, R13, and R1. R12 increased in accordance with the rising R2, whereas R13 decreased, resulting in a relatively stable resistance change. Figure 3(b) illustrates the stretching strain ε (up to 140%) applied at a stretching rate of 10.1 mm/min, along with the relative variation ratio (ΔR/R0) of resistance. The characteristic curve displays two distinct regions with gauge factor values of 2.41 (in the initial stage 0 < ε < 80%) and 7.84 (in the subsequent stage 80%–140%) resulting from the synergistic conductive paths within the resistive structure. As the strain increases into the second stage (80%–140%), the peak experiences gradual stretching, leading to the virtual disappearance of the peak valley microstructures. This result in a significant increase in R1, while R2 and R3 remain unchanged. The increased R1 leads to an increase in both R12 and R13, resulting in greater sensitivity in this stage. Expressing the sensitivity of the sensor as S = ((ΔR/R0)/Δp) with Δp denoting the applied pressure. Figure 3(c) shows the relative resistance change (ΔR/R0) under different pressures.

With the increase of pressure, the resistance component structure will stretch in the tangential direction, which leads to an increase in the change of relative resistance. However, the tension of the sensor in the tangential direction is limited, so its sensitivity is low, about 0.684 kPa−1. The slight decrease in R3 under pressure is responsible for the gradual increase observed from 0 to 2 kPa, as it restrains the resistance from rising. Figure 3(d) shows the correlation between the strain ε imposed at a stretching rate of up to 10.1 mm/min and the variation in the capacitance ratio ΔC/C0. Calculating GF requires the computation of (ΔC/C0), where C0 denotes the initial capacitance and ΔC represents the change in capacitance pre- and postapplication of the strain. As shown in Figure 3(d), the curve can be divided into two sections, with the sensor exhibiting a linear behavior in the first segment (0 < ε < 100%) before the ΔC/C0 ratio change begins to diminish. An electrical model was established to gain a better understanding of the sensor’s workings, represented by the parallel plate capacitance as C = Q/V, where the stored charge is denoted as Q and the electrostatic potential is indicated as V. Furthermore, the electric field (E) can be determined through application of Gauss’s theorem and consideration of the relative permittivity (εr) and the area (A) of the capacitor, with a calculation of E = Q/εr·A, while disregarding any fringe field effects. Since the voltage can be calculated by multiplying the electric field (E) and the distance between the electrodes (d), the definition of capacitance (C) can be shown as follows:

According to Equation 3, there is no correlation between the capacitance (C) and the stored charge (Q). Thus, any variations in the electrode plate resistance can be ignored. To validate the accuracy of the experimental results, an equation for the capacitance change ratio was formulated. The difference between the capacitance (C) under applied strain and the initial capacitance (C0) is defined as the alteration in capacitance (ΔC = C–C0). Equation 1 expresses the capacitance (C) as follows:where Δl, Δw, and Δd symbolize the size change of RCDM sensor, such as length, width, and thickness under different strains, respectively. Based on the Poisson’s ratio formula, the capacitance change (ΔC/C0) can be expressed as follows:

If the ε is used instead of Δl/l0, the ΔC/C0 can be shown as follows:where the Poisson’s ratios in the y and z directions are denoted by Vxy and Vxz, respectively. After a series of calculations, the Vxy and Vxz are 0.45 and 0.3, respectively.

The interrelation between the relative change in length (Δl/l0) and the relative change in capacitance (ΔC/C0) is visually depicted in Figure 3(d). In the first region (0 < ε < 100%), the electrode plate experiences an increase in length and area, while the distance between electrodes decreases as the stretching strain increases. This leads to an increase in capacitance with the strain and a GF value of 0.588, indicating the sensitivity to small strains. In the second region (100% < ε < 140%), the narrowing of the electrode plate becomes more prominent as the strain increases, leading to a slight change in the area of the plate. Figure 3(e) illustrates the relationship between the applied pressure and the ratio of capacitance variation (ΔC/C0). The mathematical description of the capacitance pressure sensor’s sensitivity, denoted as S, can be expressed as S = ((ΔC/C0)/Δp), where Δp represents the applied pressure and C0 indicates the initial capacitance of the sensor. This sensitivity value provides information about the ratio of change in capacitance to applied pressure and is a key performance metric for the sensor’s pressure detection capabilities. The capacitance pressure sensor sensitivity is measured to be 4.08 kPa−1. As the pressure intensifies, the distance between the plates increases, leading to a gradual rise in capacitance C. The capacitance variation ratio (ΔC/C0) is defined as follows:where λ shows the strain (Δd/d0) and the applied pressure (Δp) relationship.

The sturdiness of the RCDM sensor was evaluated by observing fluctuations in the relative resistance and capacitance (ΔR/R0, ΔC/C0) throughout four consecutive loading and unloading cycles, as depicted in Figure 4. These cycles were performed under various levels of stretching strains (20%, 50%, 80%, and 100%) and applied pressures (1, 3, and 14 kPa). The reliability of the RCDM sensor was assessed through repeated loading and unloading cycles, each of which lasted for 20 s. For stretching strains ranging from 20% to 100% and pressures ranging from 1 to 14 kPa, the measured values of ΔR/R0 and ΔC/C0 were consistently precise. These results highlight the capability of the sensor to simultaneously and accurately monitor ΔR/R0 and ΔC/C0. Furthermore, the rapid response and recovery times, as demonstrated in Figures 4(a) and 4(b), were ∼290 and 300 ms under a stretching strain of 20% and a pressure of 14 kPa, making it ideal for monitoring human motion. The ability for a sensor to be worn comfortably and without causing any discomfort is of utmost importance in order to make it widely usable. The durability of a sensor over an extended period of time is a crucial aspect that needs to be taken into consideration when assessing its overall effectiveness. Without both wearability and long-lasting ability, a sensor’s utilization is severely limited, as users may be unwilling to wear a cumbersome or short-lived device. Figures 4(c) and 4(d) show the reliability of the RCDM sensor, where the changes in ΔR/R0 and ΔC/C0 were recorded over 7,200 cycles while subjected to a 20% strain and 14 kPa pressure. The sensor exhibited consistent and stable readings, highlighting its long-term functionality and durability. As depicted in the inset of Figures 4(c) and 4(d), the ΔR/R0 and ΔC/C0 remain consistent, showcasing the sensor’s remarkable stability. Furthermore, Figures 4(e) and 4(f) illustrate the robustness of the RCDM sensor’s dynamic performance in response to various loading frequencies. The sensor’s reproducibility and stability are evident as it consistently delivers steady output in the presence of different loading frequencies. The results highlight the significance of the wearability and longevity of sensors, as it directly impacts their practical utilization in real-world applications. It has become increasingly clear that the majority of individual devices struggle with the simultaneous collection and decoupling of numerous parameters, leading to a complex and convoluted fabrication process when utilizing multiple devices. On the other hand, our RCDM sensor stands out with its outstanding decoupling capabilities, and its manufacturing process is both straightforward and economical. This provides a compelling advantage over other traditional approaches.

3.2. Application in Human Motion Monitoring and Recognition

In order to expand the application of RCDM sensors in wearable intelligent sports equipment, we installed RCDM sensors in different parts of the human body, including fingers, elbows, wrists, and knees, to monitor the changes of human posture in tennis, as illustrated in Figure 5(a1)–5(a3). As shown by the results in Figure 5(b)5(e), the protrusion of the elbow joint, accompanied by significant skin stretching, results in both high ΔR/R0 and ΔC/C0 values. Conversely, the protrusion of bones in the finger joint causes compression, leading to a relatively high ΔC/C0 and low ΔR/R0. Contrary to other joints, the wrist joint, which lacks protruding bones, experiences ΔC/C0 changes primarily due to stretch strain, which is relatively insignificant. On the other hand, skin stretching results in a significant ΔR/R0, making the wrist joint a high ΔR/R0 and low ΔC/C0 state. Finally, the knee joint displays low ΔR/R0 and ΔC/C0 values, owing to the conformal adherence of the sensor used. This close fitting of the sensor to the skin surface fails to detect sufficient strain caused by the knee’s distinctive curved shape, resulting in a low ΔR/R0-low ΔC/C0 classification.

Furthermore, to improve the RCDM sensor accuracy in the evaluation of each joint in tennis, we use the long short-term memory (LSTM) algorithm based on deep learning for the recognition training of sensing data. LSTM algorithm is a deep neural network, which can overcome the dependence on long-term data support through the use of gating unit mechanism and the regulation of sensor data. In detail, the LSTM neural network introduces the mechanism of the gating units based on the RNN [28]. As depicted in Figure 6(a), the LSTM network includes a hidden layer and two input layers for the input of sensing data, including resistance sensing data and capacitance sensing data. The output size is 10, representing 10 categories of joint postures, including wrist at 30°, 60°, and 90°; finger at 30°, 50°, and 90°; knee at 45° and 90°; and elbow at 45° and 90°. The softmax layer realizes the final classification of samples by receiving the output from the fully connected layer.

The classification performance of the RCDM sensor is assessed on a training set and a test set in a 4 : 1 ratio, with the classification accuracy as the evaluation metric. Figure 6(b)6(d) displays the confusion matrices for the classification results using only resistance data, only capacitance data, and both resistance and capacitance data. When only resistance data is utilized, the LSTM classifier fails to correctly classify wrist joints at 30° and 60°, finger joints at 30°, knee joints at 45°, and elbow joints at 45°, with classification accuracy below 87%. Using only capacitance data leads to poor performance in recognizing finger joints at 30° and 60° and elbow joints at 45°, with a considerable number of elbow joint samples at 45° being incorrectly classified as knee joints at 60° due to the similar compression level of the elbow joint at 45° sensor and the finger joint at 60° sensor, resulting in a classification accuracy lower than 60%. The results indicate that recognizing various postures using only a single capacitance or resistance signal is a challenging task. This is because the capacitance or resistance values between different poses often have a large overlap, which makes it difficult to accurately identify different poses using capacitance or resistance signals alone. Therefore, the use of dual-mode sensing technology can effectively solve this problem. Combining these two signals improves the classification accuracy of finger joint at 30°, knee joint at 45°, and elbow joint at 45° to 95.78%, 93.97%, and 92.36%, respectively, with other joint positions showing nearly 100% accuracy, which shows better performance compared with other works [29, 30]. Figure S1 shows the overall accuracy of classification of all joint postures, and shows that the highest accuracy can be achieved using these two signals. The classification accuracy of using only resistance, capacitance, and two signals is 74%, 76.93%, and 97.21%, respectively, which highlights the effectiveness of RCDM sensor in recognizing posture.

4. Conclusion

In summary, we have presented an economical and straightforward method for creating a wearable RCDM sensor with a seamless design through a casting process. Meanwhile, it can also accurately identify various joint movements by introducing the RNN. From the results, the capacitance pressure sensor sensitivity is measured to be 4.08 kPa−1. The rapid response and recovery times were ∼290 and 300 ms under a stretching strain of 20% and a pressure of 14 kPa. RCDM sensor adopts integrated structure design, which realizes the combination of resistance elements and capacitance elements. The results highlight the capability of the sensor to simultaneously and accurately monitor relative resistance change rate (ΔR/R0) and relative capacitance change rate (ΔC/C0). Also, the casting process for the production of the RCDM sensor provides a scalable and convenient method for large-scale production. With a recognition rate of 97.21% achieved through the use of deep learning algorithms, the RCDM sensor is capable of accurately identifying movement patterns and states of different joints during tennis sports. The RCDM sensor features remarkable sensitivity to strains (GF = 7.84, 0%–140%) and exceptional linearity in pressure sensitivity (S = 4.08 kPa−1) through capacitance. Furthermore, the proposed wearable multimode sensor can be produced on a large scale through the highly scalable casting process, making it suitable for a wide range of applications, including tennis sports systems. In the future, we will develop more high-performance human posture sensors to achieve their application in the field of intelligent sports.

Data Availability

All data generated or analyzed during this study are included in this published article.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Supplementary Materials

Figure S1: Histogram of classification accuracy of all joint postures using different data. (Supplementary Materials)