A Synchronized Hybrid Brain-Computer Interface System for Simultaneous Detection and Classification of Fusion EEG Signals

Yang, Dalin; Nguyen, Trung-Hau; Chung, Wan-Young

doi:https://doi.org/10.1155/2020/4137283

Complexity

On this page

Abstract Introduction Methods Results Discussion Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2020 | Article ID 4137283 | https://doi.org/10.1155/2020/4137283

A Synchronized Hybrid Brain-Computer Interface System for Simultaneous Detection and Classification of Fusion EEG Signals

Dalin Yang,¹Trung-Hau Nguyen,¹and Wan-Young Chung¹

Academic Editor: Dan Selişteanu

Received16 Mar 2020

Revised21 May 2020

Accepted25 May 2020

Published15 Jun 2020

Abstract

Brain-computer interface (BCI) technology represents a fast-growing field of research and applications for disabled and healthy people, which is a direct communication pathway to translate the neural information into an active command. Owing to the complicated headset structure, low accuracies, extended training periods, and nonstationary noises, BCI still has many challenges that should be dealt with for further facilitation of BCI technology use in daily life. In this study, a simplified synchronized hybrid BCI system is proposed for multiple command control by the electroencephalograph (EEG) signals in the motor cortex. This system can detect the single motor imagery (MI) task, single steady-state visually evoked potential (SSVEP) task, and hybrid MI + SSVEP tasks simultaneously (total ten mental tasks) via 2 EEG channels with high accuracy. The fast independent component analysis algorithm is employed to hybrid signals for obtaining clear EEG signals resulting from denoising. Feature extraction is performed by the wavelet transform, which is extracted by the features in the frequency and time domains. Furthermore, a four-layer convolutional neural network (CNN) is used as a classifier to distinguish different mental tasks. Finally, the hybrid MI + SSVEP system with a simple structure achieves a high accuracy of 95.56%. Additionally, the single MI-based and the SSVEP-based BCI system obtain the classification accuracy of 90.16% and 93.21%, respectively. Experimental results indicate that the synchronized hybrid BCI system could achieve multiple command control with a simple structure. In comparison with the single MI-based and the SSVEP-based BCI system, the hybrid MI + SSVEP BCI system shows a stable performance and higher efficiency. The proposed investigation provides a new method for the multiple command control by a hybrid BCI system. Also, the proposed BCI system offers the possibility of friendly utilization for disabled people because of its reliability, ease of use, and simplified headset structure.

1. Introduction

Brain-computer interface (BCI) provides a communication pathway to access and understand the neural activity in which the user’s intent is translated for control of the external device, such as computer, assistive applications, and neural prosthetics [1]. Generally, the noninvasive BCI is divided into four types which include functional magnetic resonance imaging (fMRI), functional near-infrared spectroscopy (fNIRS), magnetoencephalography (MEG) [2], and electroencephalograph (EEG) [3–5]. With the advantage of the temporal resolution (within millisecond), portable, and low cost, EEG is considered to be the most actively used type in BCI [6]. Unfortunately, the accuracy of classification for EEG signals decreases when the control commands increase [4]. Thus, the hybrid BCI is proposed to overcome the disadvantages mentioned above.

Currently, the hybrid BCI is defined as the combination of either more than two BCI types or two brain signals with a single BCI type [7, 8]. In the case of the hybrid BCI with multiple types, different types could provide the brain information in various aspects, in which fNIRS offers the information for hemodynamic metabolism, and EEG delivers the message of the neuron spike. However, the limitation of each type is influenced by each other (e.g., time response, portable, and cost). The hybrid BCI system of multiple brain signals with a single BCI type shows superiority due to its same sample rate and device [9, 10]. In this case, the typical brain activities for a single EEG type include motor imagery (MI) [11, 12], steady-state visually evoked potential (SSVEP) [13, 14], and P300 component of the event-related potential (ERP) [15, 16].

Hybrid EEG-based BCI systems have been popularly investigated in previous research studies [17]. There are two methods divided based on the process approach of the EEG signals: sequential mode and simultaneous mode. Most of the studies prefer the sequential mode, implying the step-by-step disposal of tasks [3], for instance, Cao et al. [18] and Ma et al. [19] analysed both MI and SSVEP signals from two independent procedures and combined the results to produce a final command for the hybrid system. For increasing classification accuracy and improve information transmission rates, the simultaneous mode is recommended for the hybrid EEG-based BCI system [20].

Generally, P300 is induced by the oddball paradigm, which regards to the endogenous potential of individuals. The amplitude of the P300 wave is weak (2–5 µV); it usually needs to record several responses to enhance the magnitude. Furthermore, the high-speed stimulation of the flashes easily leads to fatigue. MI is a mental imagining task. It could evoke the event-related desynchronization/synchronization (ERD/ERS) in the sensorimotor brain area [21, 22]. MI is widely used for device control and rehabilitation. SSVEP has shown reliable performance for discrete control techniques and has a reasonable signal-to-noise ratio produced by the fast-repetitive movement of visual stimuli. Therefore, there are a wide variety of studies [23–25] which employ the MI + SSVEP system to alternate the single systems since the MI task is the endogenous mental task that can avoid fatigue caused by long-term gazing via the SSVEP task. Besides that, the hybrid MI + SSVEP system overcomes the limitations of MI, which have fewer commands. Jin et al. [26] mentioned that MI was less apparent when stimuli tasks were added. Later on, Chio et al. [27] illustrated that employing the threshold method can alleviate interference between two brain activities. Therefore, the hybrid MI + SSVEP system is more appropriate for the system, which requires the demands of multicommands, fast response time, and high accuracy.

So far, one of the significant challenges for the BCI system is the complex design structure. It makes it challenging to utilize BCI in daily life. A simplified BCI system is essential for eradicating the problem and maintaining a good performance [28, 29]. As shown in the previous literature [28, 30], C3 and C4 electrodes (i.e., the sensorimotor cortex) can record significant characteristics during the motor imagery task. Also, it was reported [31] that SSVEP signals could be measured from whole-brain areas (e.g., occipital cortex, sensorimotor cortex, and frontal cortex). Therefore, in order to acquire the hybrid signals (i.e., MI and SSVEP) from them as less as the possible channel, C3 and C4 are selected for this study. In addition, the simplified BCI system is more requirable for a clear EEG signal and accurate classifier than a multiple-channel system. In comparison with the traditional band-pass filter, independent component analysis (ICA) shows a robust ability to separate fusion signals into their additive independent or source components. As the recent achievement of the classification method (e.g., machine learning and deep learning) in the BCI field, the traditional classifiers (i.e., linear discrimination analysis and support vector machine) explore the good performance [21, 32]. The researchers try to alternate the traditional classifiers (i.e., linear discrimination analysis and support vector machine) by deep learning methods in BCI fields. Especially, convolutional neural networks (CNNs) show a promising advantage for conducting the classification [33, 34].

To verify the feasibility of the simplified hybrid BCI system, this study adopts a simultaneous method of acquiring the fusion of MI and SSVEP signals on the two EEG channels. Totally, ten mental tasks could be recognized with high accuracy from three different EEG types. In this study, the FastICA was used to purify and denoise the interference from the fusion hybrid EEG signals. The features in the time and the feature domain were extracted by the wavelet transform (WT) algorithm. CNN was applied for classifying the different mental signals. To the best of authors’ knowledge, this study is the first of its kind, detecting simultaneous MI and SSVEP hybrid signals from the same channel based on optimized algorithms (i.e., FastICA, WT, and CNN). Compared to simple hybrid simultaneous BCI systems, the proposed system detects three different EEG types in the same CNN model by utilizing a simplified BCI structure (i.e., two channels) for multiple command (i.e., ten comments) control. This investigation verifies the feasibility of the simplified headset structure, which could be applied for the multicommand BCI control system in daily life for the disability.

2. Methods

2.1. System Parameters

Keeping with regulations and guidelines established by the local ethics committee, a total of 6 healthy adults attended this experiment, including four females and seven males, aged 23 to 38 years. Participants gave written consent before the experiments. EEG signals were recorded from the central part of the head using two Ag/AgCI electrodes supported by a high-density wireless EEG headset (Cognionics Inc., San Diego, USA). The electrode distribution and the channel number are shown in Figure 1. The impedance of all electrodes was kept below 5 kΩ, sampled at 500 Hz, and the band-pass was filtered between 1 and 50 Hz. Subjects were seated about 50 cm in front of the stimulation.

2.2. Experiment Paradiagm

A 21 inch light-emitting diode monitor with a 1920 × 1080 resolution and a 60 Hz refresh rate was used as the stimulator. Four different flickering frequencies (i.e., A: 6.6 Hz, B: 7.5 Hz, C: 8.57 Hz, and D: 9.6 Hz) were employed as a stimulus. The experiment involved ten runs for each task (i.e., A, B, C, D, A and MI, B and MI, C and MI, D and MI, idle, and MI task). Subjects were asked to sit in a comfortable chair and were instructed to follow the experimental procedure corresponding to the cues mentioned in Figure 2. The fixation period lasts 0–3 s. After 3 s, the subject executes a task based on the cue (i.e., randomly display). Each task includes ten trials. If the screen shows an MI task cue, the subject must imagine he or she is grasping his or her right hand for 5 s. If the interface shows the SSVEP cue, the participants should look at the stimulus flicker window and keep his or her body under a calm state. Then, If MI and SSVEP cues are shown on the screen, the subject must execute both tasks at the same time (i.e., thinking as if he is grasping his right hand and looking at the specific stimulator). After the 5 s execution window, users were given 2 s to relax after each run. Each task totally took 8 s. In the SSVEP tasks, the screen shows four different windows with different frequencies marked. The subjects were asked to look at a specific stimulator for each case (i.e., frequency).

Figure 2

Graphical user interface for the hybrid BCI system. (a) The caption EEG tasks shows the 10 mental tasks to be performed, which include idle state, MI task, SSVEP task, Freq. A-SSVEP task, Freq. B-SSVEP task, Freq. C-SSVEP task, Freq. D-SSVEP task, MI + Freq. A-SSVEP task, MI + Freq. B-SSVEP task, MI + Freq. C-SSVEP task, and MI + Freq. D-SSVEP task. (b) Time window for the performance of tasks divided into 2 parts for clarification, 0–3 s (rest state time) and 3–8 s (task-performing window). (c) Signal acquiring from the above performed tasks. (d) Processing, extraction, and classification of signals.

After the recording of the signals, the further processing steps are highlighted in Figure 3. Each mental task signal was processed, extracted, and classified. The procedure for signal data analysis is listed as follows:(i)Preprocess 1: used a band-pass filter (5–25 Hz) to acquire the Mu band (8–13 Hz) and SSVEP signals (6.6–9.6 Hz). Mu band was used as it is efficient for detecting the MI tasks [16]. Band-pass filter frequency range covers all the required frequencies; it helps to remove the noise and provide the desired feature.(ii)Preprocess 2: FastICA was employed to extract pure EEG signals as this method is fast and effective in obtaining the independent source signals from the fusion neural signals.(iii)Extract feature: the features of time and frequency domains were analysed by the WT since the WT could extract both time and frequency domains simultaneously. In comparison with the FFT, FFT only shows the feature in the frequency domain, which might lead to the loss of the feature in the time domain.(iv)Classifier: extracted features were the input to train the CNN model. CNN was employed for classification as it can classify multiple features by shared kernel parameters.

2.3. Fast Independent Component Analysis

In this study, we employed the FastICA algorithm to extract components, which have high-speed convergence compared to the traditional ICA. Suppose MI and SSVEP source signals are independent and non-Gaussian. In the preliminary stage, PCA is employed for feature enhancement and decorrelation. The rearrangement of the input matrix along the principal components of the data purifies the MI + SSVEP signals for the second stage. FastICA (i.e., fixed-point algorithm) is a fast optimization iterative algorithm based on batch processing [35]. This paper applies negentropy to measure the properties of non-Gaussian signals. The definition of negentropy is shown in equation (1). Here, Y_Gauss is the Gaussian random variable having a similar variance as Y. is the differential entropy for the random variable, as in equation (2).

According to information theory, the random variable with the same variance of the Gaussian distribution has the most extensive differential entropy. When Y has a Gaussian distribution of , the non-Gaussian of Y is stronger. The smaller the differential entropy (i.e., value), the higher can be used as the random variable. The calculation of the Y non-Gaussian using the definition of negative entropy of probability density distribution function needs to know Y. However, in reality, it is challenging to calculate Y. Thus, we use the following approximation formula:

refers to the formula for average computing. FastICA helps to find a dimension for WTX (y = WTX), which has the largest non-Gaussian signal. Non-Gaussian uses the approximation of the negative entropy to measure the signal. The variance of the WTX constraint is 1 for bleaching data, equal to that of the normal constraint of W to 1. Suppose is an independent source signal, is the mixed signal, and are the basis vectors from the column of the mixing matrix. As shown in equation (4), the goal is to estimate both unknowns from , with appropriate assumptions about the statistical properties of the source distributions. The solution is sought in equation (5), where B is the separating matrix.

2.4. Wavelet Transformation

In the signal processing region, Fourier transformation (FT), short-time Fourier transform (STFT), and WT are the most popular methods for feature extraction. FT is applied to obtain frequency components for different periods. STFT can get the original signal’s frequency components varied over time by making a time slice of the original signals. In the WT case, the signals can be representations of a square-integrable function by a particular series generated by a mother wavelet. As we know, the STFT window’s length is stationary, which cannot detect the frequency of unsteadily changing EEG signals in the time domain. Since the mother wavelet could make dilation and translation to adjust the represented original signals, we can quickly analyse the raw signals in the time domain and the frequency domain [36].

Equation (6) shows how to get the feature from the original signals . The parameters a and show the property of the dilation and the translation, respectively. is the function of the wavelet. Here, we use the Morlet wavelet to be as the wavelet, as shown in equation (7).

However, determining the suitable wavelet is a significant point to keep the original feature from the raw EEG signals. Since the properties of the wavelet could be known easily as a product of time range and frequency band , which can be calculated using equations (8) and (9), respectively. t₀ and ω₀ are given in equations (10) and (11). Thus, the WT algorithm can get a high resolution by the time-frequency trade-off.

2.5. Convolutional Neural Network

To avoid complicated direct pretreatment and input of the original images, CNN is widely used, including several paired convolution-pooling layers and a fully connected layer as the output. In the general neural network, the number of connected neurons is the same as input data; too much data cannot efficiently train the model. Therefore, the CNN employs local respective field and parameter sharing to reduce dimensions, input images recombined with several 2-dimensional filters in the convolution layer, and it is subsampled to a smaller size in the pooling layer. To explain the procedure, each matrix , which is achieved by the WT algorithm, is paired with the ith row and jth column. is the weight of the mth row and the nth column, and W_b is the essential item of the filter. ƒ is the activation function (i.e., ReLU function). The element of the feature map is shown in the following equation:

The time-frequency-based feature is used as the input. The network is composed of four layers, each composed of one or more neurons. The CNN topology (Figure 4) and interpretation are described as follows:

(i)Layer 0 (L0): based on the result of WT, we obtained a 3-dimensional feature matrix (time, frequency, and amplitude). After collecting all task signals with different labels, the whole of the dataset would feed the CNN to train the model.(ii)Layer 1 (L1): in the time domain, considering the dynamic feature applied to all data as the input, TS = 4,000. We select 5–15 Hz or 5–25 Hz (harmonic feature), FS = 10/20. Thus, the size of the matrix is . The input layer is with 0 ≤ i ≤ FS and 0 ≤ j ≤ TS.(iii)Layer 2 (L2): the first layer consists of the convolution and pooling layers. The convolution filter size is with the same padding, with every step equated to 1. To avoid loss from the input information sources, the max-pooling layer is employed. In this system, the max-pooling method is used with a filter size of . Thus, after processing L1, the matrix size will shrink to neurons.(iv)Layer 3 (L3): the third hidden layer is the fully connected layer, composed of 280 or 140 neurons.(v)Layer 4 (L4): the fourth hidden layer is also the fully connected layer, conducted by the ten neurons reflecting the ten commands.

3. Results

3.1. PC Configuration and Feature Analysis

In this study, the experiments were conducted in MATLAB via an Intel 3.40-GHz core i7 PC with 8 GB RAM. MATLAB was used to process the feature and perform the classification. Figure 5 shows the features of the ten mental tasks; the x-axis exhibits the time series, and the y-axis shows the frequency (unit of Hz). When the subject looks for the flicker of 6.6 Hz frequency, the EEG signals generated by the brain correspond to the respective frequency. Therefore, the WT power spectrum shows a high power for the respective frequency (i.e., 7.5 Hz, 8.57 Hz, and 9.6 Hz). When the subject performs the MI task, high power exhibits around 10–12 Hz, as shown in the WT power spectrum. The brain generates double features, while the subject executes hybrid tasks. For instance, the double features of 6.6 Hz + MI show a high-power spectrum, which appears in the frequency band of 6.6 Hz and 10–12 Hz. Therefore, the classifier could recognize various features for the performed tasks.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

3.2. Results for the Comparison of SSVEP Features

Desired to maintain a balance between the amount of the input data and the system’s performance, the experiment used two frequency windows (5–15 Hz and 5–25 Hz) to process data and to provide functional input verification for different CNN models. Moreover, the frequency window of 5–15 Hz could only detect single frequencies (i.e., 6.6 Hz, 7.5 Hz, 8.57 Hz, and. 9.6 Hz), and the 5–25 Hz window could only detect the second harmonic frequencies (i.e., 13.2 Hz, 15 Hz, 17.14 Hz, and 19.12 Hz). The 3rd harmonic frequency was too weak and unclear. Therefore, we did not consider it. Figure 6 shows a single frequency feature and the second harmonic frequency feature. Considering that MI could influence the hybrid feature, Figure 7(a) used only the SSVEP signal for comparison. Figure 7(a) represents the accuracy of the six subjects during the SSVEP tasks of the different frequency bands (i.e., 5–15 Hz and 5–25 Hz). The harmonic feature (5–25 Hz) showed better classification accuracy (95.6%) as the harmonic feature includes both the original frequency feature (6.6 Hz, 7.5 Hz, 8.57 Hz, and 9.6 Hz) and the second harmonic frequency feature (13.2 Hz, 15 Hz, 17.14 Hz, and 19.2 Hz).

(a)

(b)

(c)

(d)

(a)

(b)

3.3. Results for the Comparison of FastICA

Typically, SSVEP signals are measured at the occipital cortex areas. As the recent studies [20, 21] demonstrated, the sensorimotor cortex region can detect the SSVEP signals as well. As mentioned before, the EEG signals are acquired from the motor nerve area. To get a clear feature and reduce interference from two different fusion signals, we applied FastICA to extract MI and SSVEP signals. In this experiment, we compared the signals with and without FastICA as the input to train the CNN model. Figure 7(b) represents classification results by the signals, which employed FastICA and without FastICA. The result indicated that the signal employing FastICA achieved the highest accuracy (95.53%) compared with the one without FastICA (73.41%). Error bar shows that the classification result is stable when the FastICA algorithm processes the dataset.

3.4. Results for the Comparison of Different Classifiers

Concerning the simple MI experiment, most studies employed support vector machines (SVMs) as the classier. For the SSVEP case, the least absolute shrinkage selection operator (LASSO) and the canonical correlation analysis (CCA) algorithms are the popular classifiers to be used in BCI areas. In order to verify the advantage of the CNN for the classification, we employ the same dataset which has been processed by FastICA. The results exhibit that the classifier’s performance directly influences the accuracy of the system. Comparing with the CNN, the performance of SVM, CCA, and LASSO could not satisfy the requirement of high accuracy. After using the CNN as the classifier, the accuracy of MI and SSVEP improved by 7.29% and 6.26%, respectively. CNN method shows steadier performance than both traditional methods, as shown in Table 1. Meanwhile, the hybrid BCI system has higher accuracy than the single MI and SSVEP signals.

3.5. Results for the Single and Hybrid System

In order to exhibit the advantage of the simultaneous hybrid MI + SSVEP system, we divided the dataset to the single feature (i.e., MI/SSVEP) and hybrid feature (i.e., MI + SSVEP). Using the separate signals, the CNN model is trained, and their performances are verified. The results are shown in Figure 8. According to the results showing a loss, the hybrid system illustrated a more stable performance than the single system, especially for MI. The accuracy also improved by different degrees. Therefore, the hybrid system was the best choice for the simultaneously controlled multicommand system. The simultaneous hybrid signals could be extended for multiple commands and consequently breaking out the limitation of the number of mental tasks and the cost of time windows. The sequential hybrid system needs at least two time windows to conduct one command, whereas the simultaneous hybrid mode reduces half of the time windows. Moreover, comparing the traditional hybrid system, this system also could conduct synchronized recognition for the single MI mental task, single SSVEP single task, and hybrid MI + SSVEP task.

(a)

(b)

(c)

(d)

(e)

(f)

4. Discussion

In this study, we offer a synchronized hybrid BCI system via a simple structure to recognize single MI, SSVEP, and hybrid MI + SSVEP modes (i.e., total ten mental tasks) simultaneously with high accuracy. To the best of our knowledge, this is the first research to recognize the ten mental tasks via 2 EEG channels with good accuracy of 95.56%. The critical point for the hybrid BCI system is that the corresponding signal components should be independent and could be evoked simultaneously [16]. For the general studies, as mentioned in Table 2, [18, 19, 23, 29, 37–40] studies employed different classifiers to do the classification, whereas our system applies a single CNN classifier that could recognize all the required tasks. In order to acquire enough signals to train the CNN model, six healthy participants attended our experiment. Each subject’s signal would separate 60% as the training data; the rest of the 40% signals was divided as the test data to demonstrate the trained model. Maximizing the number of commands and minimizing the number of the EEG channel was one of our goals for this study. As demonstrated by Allison et al. [20], 32-channel performance is 2-3% higher than the 3-channels, and considering more channels also could increase the artificial noise and bring inconvenience. Since the SSVEP can be loaded in the whole of the scalp, thus, we selected channels C3 and C4, which acquire the MI and SSVEP signals at the same time. To compensate for the loss which occurs due to the use of a smaller number of channels, we applied the FastICA to obtain more purified EEG signals and used WT to extract the time and frequency features, as more the number of features, easier is to classify different EEG tasks. With reference to Table 2, the range of the commands varied from a minimum number of 2 to a maximum of 8, with an accuracy ranging from 60 to 91%, whereas our study uses ten commands and results in an accuracy of 95.56%.

5. Conclusion

This paper introduced a synchronized hybrid EEG system, based on MI and SSVEP signals, which could recognize multiple mental tasks using a simple structure of two EEG channels. FastICA is applied as a filter to purify the fusion EEG signals. Wavelet transform is employed to get the 2-dimensional feature (i.e., time and frequency domains). CNN algorithm is utilized to be the classifier for EEG signals. Results show that the proposed synchronized hybrid system can recognize three different EEG types (i.e., ten control commands) with an accuracy of 95.56%, which is higher accuracy than single MI systems (90.16%) and single SSVEP systems (93.21%). This investigation indicates that the synchronized hybrid BCI system could achieve multiple command control with a simple structure by the MI and SSVEP signals. This finding provides a novel method for the hybrid BCI system and assesses the possibility for a simple hybrid BCI structure for multiple command control.

Data Availability

The data that support the findings of this study are available on request from the corresponding author (Wan-Young Chung). The data are not publicly available due to the restrictions, e.g., they contain information that could compromise the privacy of research participants.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea Government (MSIT) (2019R1A2C1089139).

References

F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, and B. Arnaldi, “A review of classification algorithms for EEG-based brain-computer interfaces,” Journal of Neural Engineering, vol. 4, no. 2, 2007.
View at: Publisher Site | Google Scholar
A. N. Belkacem, K. Kiso, E. Uokawa, T. Goto, S. Yorifuji, and M. Hirata, “Neural processing mechanism of mental calculation based on cerebral oscillatory changes: a comparison between abacus experts and novices,” Frontiers in Human Neuroscience, vol. 14, pp. 1–13, 2020.
View at: Publisher Site | Google Scholar
S. Amiri, R. Fazel-Rezai, and V. Asadpour, “A review of hybrid brain-computer interface systems,” Advances in Human-Computer Interaction, vol. 2013, Article ID 187024, 8 pages, 2013.
View at: Publisher Site | Google Scholar
K.-S. Hong and M. J. Khan, “Hybrid brain-computer interface techniques for improved classification accuracy and increased number of commands: a review,” Frontiers in Neurorobotics, vol. 11, 2017.
View at: Publisher Site | Google Scholar
N. Yoshimura, A. Nishimoto, A. N. Belkacem et al., “Decoding of covert vowel articulation using electroencephalography cortical currents,” Frontiers in Neuroscience, vol. 10, pp. 1–15, 2016.
View at: Publisher Site | Google Scholar
A. P. Buccino, H. O. Keles, and A. Omurtag, “Hybrid EEG-fNIRS asynchronous brain-computer interface for multiple motor tasks,” PLoS One, vol. 11, no. 1, Article ID e0146610, 2016.
View at: Publisher Site | Google Scholar
G. Pfurtscheller, “The hybrid BCI,” Frontiers in Neuroscience, vol. 4, pp. 1–11, 2010.
View at: Publisher Site | Google Scholar
C. Chen, P. Zhou, A. N. Belkacem et al., “Quadcopter robot control based on hybrid brain-computer interface system,” Sensors and Materials, vol. 32, no. 3, p. 991, 2020.
View at: Publisher Site | Google Scholar
Z. Li, S. Zhang, and J. Pan, “Advances in hybrid brain-computer interfaces: principles, design, and applications,” Computational Intelligence and Neuroscience, vol. 2019, Article ID 3807670, 9 pages, 2019.
View at: Publisher Site | Google Scholar
Q. Gao, L. Dou, A. N. Belkacem, and C. Chen, “Noninvasive electroencephalogram based control of a robotic arm for writing task using hybrid BCI system,” BioMed Research International, vol. 2017, Article ID 8316485, 8 pages, 2017.
View at: Publisher Site | Google Scholar
C. R. Madan and A. Singhal, “Motor imagery and higher-level cognition: four hurdles before research can sprint forward,” Cognitive Processing, vol. 13, no. 3, pp. 211–229, 2012.
View at: Publisher Site | Google Scholar
G. Pfurtscheller, C. Neuper, D. Flotzinger, and M. Pregenzer, “EEG-based discrimination between imagination of right and left hand movement,” Electroencephalography and Clinical Neurophysiology, vol. 103, no. 6, pp. 642–651, 1997.
View at: Publisher Site | Google Scholar
D. J. McFarland and J. R. Wolpaw, “EEG-based brain-computer interfaces,” Current Opinion in Biomedical Engineering, vol. 4, pp. 194–200, 2017.
View at: Publisher Site | Google Scholar
L. Shao, L. Zhang, A. N. Belkacem et al., “EEG-controlled wall-crawling cleaning robot using SSVEP-based brain-computer interface,” Journal of Healthcare Engineering, vol. 2020, Article ID 6968713, 11 pages, 2020.
View at: Publisher Site | Google Scholar
R. Corralejo, L. F. Nicolás-Alonso, D. Álvarez, and R. Hornero, “A P300-based brain-computer interface aimed at operating electronic devices at home for severely disabled people,” Medical & Biological Engineering & Computing, vol. 52, no. 10, pp. 861–872, 2014.
View at: Publisher Site | Google Scholar
E. Donchin, K. M. Spencer, and R. Wijesinghe, “The mental prosthesis: assessing the speed of a P300-based brain-computer interface,” IEEE Transactions on Rehabilitation Engineering, vol. 8, no. 2, pp. 174–179, 2000.
View at: Publisher Site | Google Scholar
R. A. Ramadan, S. Refat, M. A. Elshahed, and R. A. Ali, “Basics of brain computer interface,” in Brain-Computer Interfaces, vol. 74, pp. 31–50, Springer, Berlin, Germany, 2015.
View at: Publisher Site | Google Scholar
L. Cao, J. Li, H. Ji, and C. Jiang, “A hybrid brain computer interface system based on the neurophysiological protocol and brain-actuated switch for wheelchair control,” Journal of Neuroscience Methods, vol. 229, pp. 33–43, 2014.
View at: Publisher Site | Google Scholar
T. Ma, H. Li, L. Deng et al., “The hybrid BCI system for movement control by combining motor imagery and moving onset visual evoked potential,” Journal of Neural Engineering, vol. 14, no. 2, Article ID 026015, 2017.
View at: Publisher Site | Google Scholar
B. Z. Allison, C. Brunner, V. Kaiser, G. R. Müller-Putz, C. Neuper, and G. Pfurtscheller, “Toward a hybrid brain-computer interface based on imagined movement and visual attention,” Journal of Neural Engineering, vol. 7, no. 2, Article ID 026007, 2010.
View at: Publisher Site | Google Scholar
E. Dong, C. Li, L. Li, S. Du, A. N. Belkacem, and C. Chen, “Classification of multi-class motor imagery with a novel hierarchical SVM algorithm for brain-computer interfaces,” Medical & Biological Engineering & Computing, vol. 55, no. 10, pp. 1809–1818, 2017.
View at: Publisher Site | Google Scholar
A. N. Belkacem, S. Nishio, T. Suzuki, H. Ishiguro, and M. Hirata, “Neuromagnetic decoding of simultaneous bilateral hand movements for multidimensional brain-machine interfaces,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 26, no. 6, pp. 1301–1310, 2018.
View at: Publisher Site | Google Scholar
P. Horki, T. Solis-Escalante, C. Neuper, and G. Müller-Putz, “Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb,” Medical & Biological Engineering & Computing, vol. 49, no. 5, pp. 567–577, 2011.
View at: Publisher Site | Google Scholar
O. Dehzangi, Y. Zou, and R. Jafari, “Simultaneous classification of motor imagery and SSVEP EEG signals,” in Proceedings of the 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 1303–1306, San Diego, CA, USA, November 2013.
View at: Publisher Site | Google Scholar
J. Guan, Z. Zhang, R. Liu, Z. Chen, and F. Duan, “The development of a brain-controlled lock based on SSVEP and MI,” Lecture Notes in Electrical Engineering, vol. 459, pp. 407–416, 2018.
View at: Publisher Site | Google Scholar
J. Jin, I. Daly, Y. Zhang, X. Wang, and A. Cichocki, “An optimized ERP brain-computer interface based on facial expression changes,” Journal of Neural Engineering, vol. 11, no. 3, Article ID 036004, 2014.
View at: Publisher Site | Google Scholar
I. Choi, I. Rhiu, Y. Lee, M. H. Yun, and C. S. Nam, “A systematic review of hybrid brain-computer interfaces: Taxonomy and usability perspectives,” PLoS One, vol. 12, no. 4, Article ID e0176674, 2017.
View at: Publisher Site | Google Scholar
S. Ge, R. Wang, and D. Yu, “Classification of four-class motor imagery employing single-channel electroencephalography,” PLoS One, vol. 9, no. 6, Article ID e98019, 2014.
View at: Publisher Site | Google Scholar
L.-W. Ko, S. S. K. Ranga, O. Komarov, and C.-C. Chen, “Development of single-channel hybrid BCI system using motor imagery and SSVEP,” Journal of Healthcare Engineering, vol. 2017, Article ID 3789386, 7 pages, 2017.
View at: Publisher Site | Google Scholar
G. Pfurtscheller, C. Brunner, A. Schlögl, and F. H. Lopes da Silva, “Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks,” Neuroimage, vol. 31, no. 1, pp. 153–159, 2006.
View at: Publisher Site | Google Scholar
Y. Te Wang, Y. Wang, C. K. Cheng, and T. P. Jung, “Measuring steady-state visual evoked potentials from non-hair-bearing areas,” in Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1806–1809, San Diego, CA, USA, November 2012.
View at: Publisher Site | Google Scholar
A. N. Belkacem, H. Hirose, N. Yoshimuraand, and D. Shin, “Classification of four eye directions from EEG signals for eye-movement-based communication systems,” Journal of Medical and Biological Engineering, vol. 34, no. 6, pp. 581–588, 2013.
View at: Publisher Site | Google Scholar
F. Lotte, L. Bougrain, A. Cichocki et al., “A review of classification algorithms for EEG-based brain–computer interfaces : a 10 year update,” Journal of Neural Engineering, vol. 15, no. 3, 2018.
View at: Publisher Site | Google Scholar
P. Mirowski, D. Madhavan, Y. LeCun, and R. Kuzniecky, “Classification of patterns of EEG synchronization for seizure prediction,” Clinical Neurophysiology, vol. 120, no. 11, pp. 1927–1940, 2009.
View at: Publisher Site | Google Scholar
M. Zhang, M. Zhu, and W. Ma, “Implementation of FastICA on DSP for blind source separation,” Procedia Engineering, vol. 29, no. 6, pp. 4228–4233, 2012.
View at: Publisher Site | Google Scholar
V. D. Hoang, “Wavelet-based spectral analysis,” TrAC Trends in Analytical Chemistry, vol. 62, pp. 144–153, 2014.
View at: Publisher Site | Google Scholar
J. D. R. Millán, ““Combining brain–computer interfaces and assistive technologies: state-of-the-art and challenges,” Frontiers in Neuroscience, vol. 1, p. 161, 2010.
View at: Publisher Site | Google Scholar
J. Long, Y. Li, H. Wang, T. Yu, J. Pan, and F. Li, “A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair,” IEEE Transactions on Neural Systems and Rehabilitation Engineering: A Publication of the IEEE Engineering in Medicine and Biology Society, vol. 20, no. 5, pp. 720–729, 2012.
View at: Publisher Site | Google Scholar
B. Z. Allison, C. Brunner, C. Altstätter, I. C. Wagner, S. Grissmann, and C. Neuper, “A hybrid ERD/SSVEP BCI for continuous simultaneous two dimensional cursor control,” Journal of Neuroscience Methods, vol. 209, no. 2, pp. 299–307, 2012.
View at: Publisher Site | Google Scholar
J. Li, H. Ji, L. Cao et al., “Evaluation and application of a hybrid brain computer interface for real wheelchair parallel control with multi-degree of freedom,” International Journal of Neural Systems, vol. 24, no. 04, Article ID 1450014, 2014.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2020 Dalin Yang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies