Abstract
In sports science, the automation of performance analysis and assessment is urgently required to increase the evaluation accuracy and decrease the performance analysis time of a subject. Existing methods of performance analysis and assessment are either performed manually based on human experts’ opinions or using motion analysis software, i.e., biomechanical analysis software, to assess only one side of a subject. Therefore, we propose an automated system for performance analysis and assessment that can be used for any human movement. The performance of any skill can be described by a curve depicting the joint angle over the time required to perform a skill. In this study, we focus on only 14 body joints, and each joint comprises three angles. The proposed system comprises three main stages. In the first stage, data are obtained using motion capture inertial measurement unit sensors from top professional fighters/players while they are performing a certain skill. In the second stage, the collected sensor data obtained are input to the biomechanical software to extract the player’s joint angle curve. Finally, each joint angle curve is processed using a continuous wavelet transform to extract the main curve points (i.e., peaks and valleys). Finally, after extracting the joint curves from several top players, we summarize the players’ curves based on five statistical indicators, i.e., the minimum, maximum, mean, and mean standard deviation. These five summarized curves are regarded as standard performance curves for the joint angle. When a player’s joint curve is surrounded by the five summarized curves, the performance is considered acceptable. Otherwise, the performance is considered unsatisfactory. The proposed system is evaluated based on four different karate skills. The results of the proposed system are identical to the decisions of the expert panels and are thus suitable for real-time decisions.
1. Introduction
Scientific and technical advancements are currently being pursued actively. Sports are an area that has developed and evolved as a result of contemporary technological advancements. The evolution of training/measurement instruments and training programs necessitate the use of efficient automated approaches to facilitate the investigation and improvement of performance assessments and analysis. A review was conducted by a committee of governors, which included a rug’s head, in the language of the International Rules of Arbitration for Karate. However, relying entirely on human observation and expertise is inadequate for assessing player performance and identifying flaws. The criteria used by judges are based on their own assessment, their personal experience, and their judgment of their performance assessment, which is a type of self-evaluation. An automated method for analyzing and comprehending skill performance as well as for identifying defects and weaknesses must be developed. This method would allow the performance analysis and training processes to be easily reviewed and developed.
Examination of human movement via biomechanics has emerged as a method for gaining knowledge and information regarding the rules and foundations of performance for certain sport activities. Generating a general analytical sportive consensus remains challenging because scientific researchers in related fields are hindered by the diversity of methodologies. These challenges emerge because of the competitiveness and confidentiality of any professional sports in which the analytic techniques, data, and tools are concealed [1]. Furthermore, the effort by researchers of sport analytics is not sustained [1]. Thus, studies have been conducted to analyze the efficiency of sport betting markets [2, 3].
By relaying on computer science methods, the performance of sport analytics can be enhanced significantly [4–7]. Specifically, machine learning and deep learning methods have been used considerably in the recognition of human activities [8–11]. For instance, neural networks have been employed in various applications over the recent decades, including rating National Collegiate Athletic Association (NCAA) college football [12], cricket performance prediction [13], and predicting National Football League (NFL) winners [14]. In the same context, the authors in [15] discussed the role of machine learning and wearable devices in various sport applications which resulted in great impact on the athlete’s performance in terms of planning, decrease the injury risks, and improve the athlete’s performances. Furthermore, wearable biofeedback sensors suit to retrieve the joint angles during water activities have been suggested in [16].
Wearable sensors and applications have been used to monitor sports performance during training and player selection [17–19]. In [17], the authors utilized a body accelerometer equipment to track ice hockey players’ skating sprints. Two accelerometers are placed on the players’ skates and waists to measure stride time, sprint phases, and sprint performance. A visualization application has been proposed for analyzing the performance of a soccer team in [18]. This application permits the visualization of imported real data. Another study presented an analysis system that employs computer vision technology in [19]. To provide an efficient evaluation, the player’s postures have been gathered and analyzed using a camera and deep learning algorithms. The absence of reliable data gathering from professional athletes is one of the limitations that prevent conducting accurate and precise evaluation of sport activities. In addition, the lack of suggesting a clear framework to objectively assess the trainees’ performance is another disadvantage.
As one of the powerful computer science algorithms, continuous wave transform (CWT) was used in several motion analysis problems. For instance, the authors in [20] utilized the CWT to analyze a frequency modulated signal, which is extracted from an input video frames. The purpose of CWT analysis is to segment the body motion and video classification. The authors in [21] proposed extracting body motion from the brain signals with the help of electroencephalography (EEG). The authors utilized the CWT method to generate spectrogram for each hand finger from the slow cortical potentials (SCPs) in the low frequency. Then, the results of using CWT show distinctive amplitude and phase patterns.
The advances of current the biomechanics of bodies’ software, wearable sensors for motion captures, and CWT method for motion data analysis encourage this work. The work includes utilizing the wearable sensors to capture the player’s motion. Then, the sensors’ data will be analyzed using biomechanics of bodies’ software to produce joints’ angles during performing a sport skill. Finally, the joints’ angles will be analyzed by the CWT method to extract the important patterns to assess the quality level of the performed sport skills. Those three phases are gathered in an online automatic assessment system. Moreover, to validate the proposed system, a group of karate player from the Egyptian men’s national team performed four skills. The collected motion data of this team were utilized to build a standard pattern for those four skills. Then, a trainee’s performance for those four karate skills will be compared to the standard patterns to assess the level of the trainee’s performance. The main contributions of this study can be summarized as follows:(1)An automated performance evaluation system for performed sport skills is proposed.(2)To the best of the authors’ knowledge, this is the first study that uses CWT in an algorithm for detecting peaks and valleys to analyze athletes’ performance.(3)Inertial measurement unit (IMU) sensors are used in conjunction with the CWT in a novel manner.(4)A new dataset is provided for four different karate skills: Barai (i.e., downward block), Oi-zuki (i.e., Lunge punch), Jodan Age–uke (i.e., upper block against head attack), and Soto–uke (i.e., outside inward) ((https://github.com/Ahmed-Fathalla/Real-Time-and-Automatic-System-for-Performance-Evaluation-of-Karate-Skills-Using-Motion-Capture-Sens)).(5)The proposed performance analysis system is examined and validated using a realistic dataset.(6)The proposed model can be easily extended to any other sport skill(s).
The rest of this paper is organized as follows: In Section 2, we exposed the essential background for the proposed work. Section 3 discusses the proposed methods for the automatic athletes’ performance analysis. The results are exposed and discussed in Section 4. Finally, the paper is concluded in Section 5.
2. Background
2.1. Continuous Wavelet Transform
The CWT has recently gained popularity in the signal analysis area. Nenadic and Burdick [22] used a 2D CWT to identify points in neural recordings, which they found to be effective. A thresholding function was used at each stage to search for local maxima, which were then recorded. A “contiguous region” was formed by joining all of the spots of local maxima together to produce a “contiguous region” after which peaks were sought within that region. Another example is the MassSpecWavelet detection technique developed by Du et al. [23], one of its disadvantages is that it cannot discriminate between single-sided slopes (i.e., step functions) and peaks in the data. On the other hand, its strength lies in its peak identification approach, which involves linking local wavelet maxima to sequences that converge to a peak.
The CWT is performed by a wavelet proportional to the negative first derivative of the Gaussian function, according to the novel method named Ridger, which is based on Du et al. [23]. Ridger conducts the CWT using a wavelet proportional to the negative first derivative of the Gaussian function. The CWT coefficients of a signal peak create a local maximum and minimum pair around the peak region, which is a wavelet-specific trait that the method uses. A further characteristic of wavelets is that when the wavelet scale is reduced, there is always a series of connected local maximum and minimum points that converge to the peak. This trait is used to locate the peak.
Using daily consumption load curves as functional data, the authors of [24] employs elastic shape analytic methods to cluster and evaluate load curves based on their forms. The critical concept is to use an elastic form metric to time warp load curves in order to align their peaks and valleys. This process eliminates phase variability from the data and enables the focus on the load profile forms. As a general principle, the alignment process aims to align the peaks and valleys of power consumption functions (load curves) by warping the time domain to get the distinctive form of the functions. This stage is referred to as the “phase-shape separation of functions” in certain circles.
To put it another way, it breaks down complicated variability in data obtained into two more manageable components: phase and shape. The vertical aspects of a function (peaks, valleys, and heights) that are invariant to deformations of the time domain are referred to as the function’s shape. The phase variability of functions, on the other hand, records variations in the placement of these characteristics when compared to a template function in a given time interval. Consequently, after eliminating the phase variability from the equations, the shape distance is the relevant distance to examine when comparing the characteristics of distinct functions.
A wavelet is a function in that meets the following conditions [25, 26]:
A CWT on a function at the position is denoted by the following notationwhere is converted by and reduced by as follows:
It is possible to utilize the CWT coefficients produced to determine the strength and position of peaks in that are of comparable magnitude to those in by looking for patterns of peaks and valleys. Changing the value of in will result in different-width wavelets, and as a result, all peaks in may be identified, independently of their width.
2.2. Dynamic Time Warping
The dynamic time warping (DTW) algorithm is a critical mean for capturing the time-varying features for multivariate time series of changing length and phase [27, 28]. This algorithm can handle multivariate time series (raw data) without requiring any prior statistical assumptions about how the data was generated. In other words, DTW is a method of comparing two temporal sequences that aren’t perfectly synchronized in order to determine the best match between them. The ability to stretch or compress the patterns of two objects locally to obtain similarity among the compared series is a significant advantage of this technique. Additionally, it is regarded as more malleable than Euclidean distance [28]. Given the following two sequences, X and Y:
The alignment between x[i] and y[j] is represented by each point coordinate (i, j). The following equation can be used to calculate the optimal path to a point :where is the Euclidean distance and is the overall path cost.
Regardless of the superior benefits of the DTW algorithm, the number of possible warping paths should be limited for efficiency reasons. The boundary, monotonicity, and continuity conditions are three constraints that can be used to accomplish this. The warping window and slope conditions are two less common conditions for an allowable warping path that should be considered. The first constrains the number of points that can fall within a given warping window, while the second regulates the slope to prevent extreme movements in one direction.
Numerous applications have benefited from the DTW algorithm; those applications include increasing the accuracy of connected digit recognition [29], which creates a reference string by concatenating the reference contours of the digits and comparing it to the test string. Selecting the gesture candidates for recognizing real-time hand gestures of a Kinect sensor is another application of the dynamic time warping [30, 31]. Additional uses involve identifying Alzheimer disease by comparing foot movements [32, 33], finding the imputation of missing values for univariate time series data [34], and differentiating bee propolis based on its geographical origin [35].
3. Methodology
3.1. System Overview
Developing a standard model for recording the professional skills of karate trainees and evaluating their competence and professionalism are two aspects of the proposed system. The system begins by recognizing the performance of a karate player from the Egyptian men’s national team. The selected player was instructed to wear the IMUs such that his movements can be monitored, and his performance was recorded in a calc file, which is a file with calc extension, as numerical data. The Axis Neuron software was used to interpret the information from the IMU sensors to determine the position and direction of the recorded motions. Thereafter, biomechanical analysis was performed using the the biomechanics of bodies (BoB) to extract the joint angles of 42 body segments. Our proposed system uses the CWT algorithm to identify the statistical information obtained for the joint angles of 42 body segments. The system creates a professional movement benchmark for each karate action, which is categorized from minimum to maximum based on the professional level. Figure 1 shows the flow system architecture of the proposed system for the development of a standard model.
After establishing the standard model, nonprofessional karate trainees were instructed wear the IMUs; subsequently, the sensor data were retrieved to evaluate their karate skills using our proposed system. The system results reflected whether the trainees’ performance was satisfactory. Figure 2 illustrates the procedure for validating each joint angle (https://youtu.be/jGf3JOyutpU). In the following subsection, the process of data acquisition will be discussed.
3.2. Dataset Acquisition
The players’ movements were monitored and recorded using a motion tracking system that uses the IMUs of the Perception Neuron Pro Edition. This system can monitor the human body motion using 17 sensors connected by a wireless network. All sensors are wirelessly attached to a computer via a USB hub connector, which simplifies the process of data entry into the device. Subsequently, the data obtained from the IMUs were automatically input to the accompanying software installed on the device. The Axis Neuron program (https://neuronmocap.com/content/axis-neuron) performs instance acquisition and data recording from the sensors attached on the players’ body segments. Based on the output data of the biomechanical analysis, the Axis Neuron software provides quantitative and descriptive outputs in digital or graphic images.
The Axis Neuron software provides users with complete control over sensors and allows them to perform four tasks: (1) Compass calibration, (2) physical sensor calibration, (3) starting and stopping the motion recording process, and (4) displaying the position and directions of motions being recorded. Additionally, it allows the users to export data in various software formats for use in kinetic analysis programs and to create three-dimensional (3D) models.
The IMUs for the motion capturing/monitoring system are composed of three internal devices: an accelerometer, a direction finder, and a magnetometer. During the calibration process, these IMUs reduce the electromagnetic wave interference, improve the accuracy of rapid movement measurements, and unify the reference directions of all sensors. Consequently, a precise measurement process is guaranteed. The lightweightness of the installed sensors, which weigh approximately 170 g (i.e., 10 g each), is advantageous to this system. Hence, the sensors in this system are distributed evenly across the players’ body segments and thus do not affect their performance. Additionally, this system is distinguished by the availability of multiple patterns for recording human body movements, including a single arm pattern, a pattern for the upper half of the human body, and a full body pattern.
3.3. The Utilized Tools
The utilized tools of the proposed systems are from three different directions. First, the data is gathered from the sensory field; then, the sensory data are analyzed using the a biomechanical tool to obtain the body joint angle values for a certain movement, and finally, these angles are processed using statistical and computer science methods.
The data gathering is completed using the Axis Neuron and the 17 body segments sensors. Later on, BoB program (https://www.bob-biomechanics.com/) works on processing the data gathered by the Axis Neuron to be processed and analyzed. The BoB software converts the sensors reading into body joint angle over the motion time. Then, we proposed extracting the aforementioned 42 body segments joint angles’ curves.
The purpose of the data should be defined once the data has been properly acquired through sensors that convey the players’ bodily movement. Motion data is crucial in evaluating sports performance. We proposed utilizing the statistical analysis to examine the players’ tactics. Thus, an effective means of measurement can be offered to assist trainers and trainees to evaluate the personal performance objectively and with digital data. At this point, a player’s technical level can be increased based on a scientific foundation along with the quality of the training process. Furthermore, a player’s own level of performance can be easily compared with that of other professional players. The minimum, maximum, mean, and standard deviation metrics were employed in the calibration model building and performance evaluation in this study. This is because the collected motion data are obtained from several high-ranked players. Thus, these players’ data should be summarized. Finally, the key points of the joint angle curve are processed using CWT to extract the key points (i.e., peaks and valleys) of the curve to be considered for the comparison issue. The proposed utilized algorithm for building the standard model to describe the optimal performance and its time complexity analysis is provided in the following subsection.
3.4. The Proposed Algorithm
Because we are primarily interested in monitoring the movements of the trainee’s joints, the peaks and valleys were extracted from the curve of the body joint angles over the duration within which a skill was performed. The optimal curving model was constructed based on a summary of movement monitoring for the best players in certain sports. The model includes the maximum, minimum, mean, and the mean standard deviation mean curves. The best zone was between the curves of the standard deviation and mean. We compared the extracted curve for the trainee’s joint movement with the optimal curve to verify the position of the generated curve. Based on the new curve position, the trainee movements were judged as either acceptable or unsatisfactory.
Algorithm 1 lists the procedures for developing a standard model to describe the optimal performance of a sport skill in terms of the CWT and summary statistics. The main body of the algorithm comprises three nested loops, which represent the number of players , number of joints, and number of joint angles. As the number of body joints of interest is 14, and each joint comprises only three angles, the only variable in these three loops appears in the first loop (i.e., the number of players). represents the number of high-ranked professional players whose performance will be used to develop the standard skill model. The time complexity of the CWT is [36], where represents the number of time points of the preformed skill. Thus, the time complexity of the first three loops is .
|
4. Experimental Results
4.1. Dataset
A descriptive technique was employed to generate the dataset. To validate this study, a group of players from the Egyptian Karate Federation was recruited and separated into two groups to perform two studies, i.e., a descriptive study (which facilitates in the developing the standard model) and an experimental study. Additionally, the players participated in an exploratory study. The players who participated in the standard model were the Egyptian men’s national team players who performed the skills investigated in the current study. Four skills were investigated for the karate sport in this study: Gedan Barai (i.e., downward block), Oi-zuki (i.e., lunge punch), Jodan Age–uke (i.e., upper block against head attack), and Soto–uke (i.e., outside inward).
In the proposed dataset, the movements of the trainee were captured and monitored using a motion tracking system by deploying IMUs. These IMUs are of the Perception Neuron Pro Edition, which includes 17 sensors for precisely monitoring human body movements [1]. The resulting data were input to a computer via a USB hub connector, and the sensors were connected to a wireless network. The data entered were interpreted using the Axis Neuron program to obtain the position and direction of movement recording and review. The exported data were recently used for kinetic analysis and 3D modeling programs.
The research sample was a skill sample that included 178 correct skill performances for all research skills, performed by 3-4 players for each independent skill by the Egyptian men’s karate national team, as shown in Table 1, which displays the number of skill performances for the skills being investigated, as well as the number of players who performed the skill.
Movement monitoring was achieved using a motion tracking system comprising IMUs (Perception Neuron Pro Edition). The system uses 17 sensors connected to a wireless network and a USB hub to connect to a computer to monitor and record human movements. The system’s operating program, Axis Neuron, developed by the system’s manufacturer, interprets these data and provides the user complete control over the system’s operation via compass calibration, physical adjustments to the sensors, and the initiation and termination of the movement recording process. Additionally, it allows the users to export data in various software formats for kinetic analysis and 3D modeling.
The system’s sensors were installed seamlessly on the human body to prevent them from obstructing the player’s movement during performance. Furthermore, the lightweightness of the system (each sensor weighs only 10 g) and the system’s total weight of 170 g distributed across various body connections did not inconvenience the players.
4.2. Justification for Selecting the Samples
The skill performances selected are those required in karate-belt advancement examinations. They can be used alone, in conjunction with other talents, or in combination with each other. Furthermore, they are the most typically employed in karate performances, where they are combined with one another and with other skills in the “Gohon Kumite” conflict. Meanwhile, the human samples selected for developing our model were members of the Egyptian men’s national team players who achieved advanced positions and global “collective-individual” numbers. They are healthy and uninjured players who are professionals in performing the skills investigated, and they demonstrate the highest level of technical performance. Finally, they agreed to participate in the study because they appreciated the study’s significance. In the following subsections, the experimental setup and results discussion will be provided.
4.3. Setup
The experimental setup consists of two stages. First, the data collection from the trainee while they were performed the karate skills (see Figure 3). Second, the computer science algorithms to analyze and process the collected data. The data collection is achieved using the perception neuron 3 sensors suit (https://neuronmocap.com/perception-neuron-3-motion-capture-system). Then, the IMUs data are converted to angles using the BoB software (https://www.bob-biomechanics.com/). For the computer science methods, we utilized the Python 3.8 programming language alongside some online packages, namely, Pandas [37], Numpy [38], Matplotlib [39], and SciPy (find_peaks_cwt) [40]. The experiments were performed on a computer running Windows 10 OS with 16-GB RAM, and 12 CPU.
4.4. Results and Discussion
The results of the proposed method can be summarized in terms of three aspects: (1) The execution times of the algorithms, (2) the obtained models, and (3) the model validation.
4.4.1. Execution Times
Tables 2 and 3 list the execution times of the main steps for both model creation and validation. These two tables list the total execution times in seconds for each main step of the proposed method in addition to the average time for executing each step once along with its standard deviation. The reported execution times show that the proposed system can provide real-time performance analysis and assessment decisions.
4.4.2. Normalization Step Evaluation
The model was developed using normalized data. The data obtained must be normalized such that they are applicable to the different performance times of different trainees. Normalization guarantees that all data obtained for one skill have the same number of data points. To evaluate the proposed normalized curve method, we selected a sample data for the plantar dorsi angle of the left ankle; the original curve for this set of data contains 140 data points (i.e., IMU readings) for the entire skill, whereas the normalized curve contains only 100 data points, as shown in Figure 4(a). A comparison between the original and normalized curves is shown in Figure 4. The evaluation involved using the DTW algorithm to visually verify the similarity between the two curves (i.e., the original and normalized curves). Figure 4(b) shows a perfect alignment, where the main points of the two curves are perfectly connected to each other. Another evaluation aspect involves the distance and cost matrices, as shown in Figure 4(d). The distance/dissimilarity matrix measures the distance between each point of the two curves. The y-axis represents the normalized curve with 100 data points, and the x-axis represents the original curve with 140 data points. The smaller the pixel, the closer is the distance, and vice versa. The distance matrix, as shown in Figure 4(d), indicates good performance, where the dark points represent the 40 points absent from the original curve. The behavior exhibited by the cost matrix is similar to that by the distance matrix. Then, the obtained standard model will be evaluated in the next subsection.
(a)
(b)
(c)
(d)
4.4.3. Obtained Model Evaluation
Finally, the obtained model was evaluated by comparing its parameters based on two sets: a set of wrongly performed skills, and a set of perfectly performed skills. The model obtained for the plantar dorsal angle of the left ankle is shown in Figure 5(a). Figure 5(b) shows the behavior of a perfectly performed skill (i.e., the red curve). The perfect performance is captured appropriately by the model curves; thus, it is classified as a correct skill. Similarly, an angle of two trails are compared in Figure 6, where the correct trial angle curve is depicted in Figure 6(a) and the incorrect trial angle curve is depicted in Figure 6(b).
(a)
(b)
(a)
(b)
Finally, the performance of the trainee was evaluated in the decision-making step. The analytical reports generated by the proposed system for assessing the players’ performance were used to create reports for aiding coaches, talent scouts, and sponsors in making decisions. Hence, evaluation was regarded as the ultimate approach to ensure the high quality and validity of decision-making. A coach can categorize players as key or nonkey players based on the decisions of the players. Determining the best playing position for a player based on his skills is another form of decision-making. In addition, the coach or trainee may utilize the player’s improvement information during training to support the training program. The outcomes of the training program’s assessment procedure may be utilized to validate the success of the selected training approach.
The positive implications of the proposed system are that it can be used by karate trainers to improve their trainees’ performance, as the proposed system can provide an analysis of how much the trainees performance is deviating from the standard performance. This analysis is per body joint, as the trainees might perform the skill correctly by one joint and wrongly by another. Moreover, keeping the historical data of the trainees can help their coaches to check the progress of the trainees’ performance. The limitations of the proposed system are discussed in the following subsection.
4.5. Limitations
The proposed system has two limitations which the missing values of the collected data. First, while collecting the data from the athlete using the wearable devices, some readings were missing due to the connectivity or other signal barriers. These trials with missing readings are discarded. Seconds, some readings are abnormal due to the effect of metal objects that negatively affecting the sensors’ accuracy, and as a result affecting the model accuracy, as discussed in [41]. Again, these trials are discarded. Thus, predicting the missing or abnormal readings will reduce the number of discarded trials. This is the future work of the proposed method.
5. Conclusions
In this study, a performance evaluation system for sport skills was proposed. The proposed system combines three different fields to automate biomechanical performance analysis for sport skills. The fields utilized were motion capture using IMU sensors, biomechanical software, and computer science algorithms (i.e., CWT and DTW). The final output of the proposed system was a set of five curves (i.e., minimum, maximum, mean, and mean±standard deviation) for each body joint angle describing perfect performance, where the curve depicts the joint angles over time. Readings from the IMU sensors were input to the biomechanical software, which subsequently converted the sensor readings into joint angle curves. This process was repeated for a set of high-ranking professional players. Subsequently, computer science algorithms were executed by selecting key points on the curves and summarizing the players’ curves into only the aforementioned five curves for each body joint angle. The proposed system automatically compress the performance of the new trainee’s motion curves against the obtained summarized curves and reports whether the new trainee’s motion curves reflect the standard performance for each body joint angle. The proposed system was validated based on four different karate skills, and the evaluation results of the proposed system were identical to those of a panel of human experts in karate.
Data Availability
The dataset used to support the findings of this study have been deposited in the github repository (https://github.com/Ahmed-Fathalla/Real-Time-and-Automatic-System-for-Performance-Evaluation-of-Karate-Skills-Using-Motion-Capture-Sens).
Conflicts of Interest
The authors declare that they have no conflicts of interest to report regarding the present study.
Authors’ Contributions
Ahmed Fathalla, Ahmed Ali, Ahmad Salah, Mahmoud Bekhit, and Esraa Eldesouky contributed equally to this work.