02 Overview of hybrid control methods based on EMG
The basic idea of the hybrid control interface based on EEG-EMG is to fuse EEG and EMG signals in the control method. The fusion of signals can be done in many different ways and may depend on factors such as specific applications and user capabilities. In this hybrid interface, combining EEG signals and EMG signals, the application of the hybrid method may vary, from a simple game control application to a prosthetic arm control application.
The main purpose of this review is to study the applications of biorobots, such as prostheses and exoskeletons, so the scope is narrowed to the application of hybrid EEG-EMG methods in biorobots. As mentioned earlier, there are many possible ways to combine EMG and EEG signals in a specific control method to increase effectiveness.
Generally speaking, EEG or EMG signals can be used to operate various parts of the application, such as components in auxiliary equipment, or all of these can be combined. The latter will allow users to smoothly switch from one control signal to another according to their preferences.
There are several methods that can be used to classify EEG-EMG hybrid control methods in biorobot applications, such as specific applications/devices (such as prostheses, exoskeletons, wheelchairs) or input processing methods. As a dual-input system, the hybrid EEG-EMG interface can process input signals simultaneously or sequentially.
In this review article, we will divide each study of hybrid control methods in biorobot applications into two categories. According to whether the input processing method is simultaneous or sequential, the comparison of EEG-EMG methods is different from the one discussed in this article. The important characteristics of the hybrid method are summarized in Table 1. What is important is that no matter what the fusion method of EEG-EMG signals is, the hybrid method can obtain higher effectiveness than the method of using EMG or EEG signals alone.
Table 1 Comparison of several different EMG-EEG hybrid methods.
Few research reports use hybrid methods to compensate for the problems in the application of EMG-based control methods in biorobot applications. EMG signals are often used as prostheses and exoskeletons. Or the control input of wheelchairs and other equipment, because sometimes the muscles of the disabled, the infirm, the elderly or the injured still have residual activity, there are still some problems to be solved in the control method based on the EMG signal.
One problem that may be encountered when using the EMG signal alone is muscle fatigue, which, in addition to the normal level of muscle contraction, also affects the amplitude and frequency spectrum of the EMG. Especially with the increase of age, the size of skeletal muscle fibers becomes smaller and the strength is weakened, which leads to the tendency of the elderly to lose strength and fatigue rapidly. The physical and mental conditions of these elderly people are changing throughout the day. In some cases, muscle fatigue may occur due to physical fatigue. In this case, it is necessary to design a control method based on electromyography to deal with muscle fatigue. influences.
Except for some attempts, there are few research reports to develop control methods based on EMG. These methods are robust to muscle fatigue. On the other hand, in the control method, the EEG signal can be used as an additional input signal. Deal with muscle fatigue, instead of relying solely on EMG signals.
It has been reported that this attempt to fuse muscle and brain signals into a hybrid BCI, according to the user’s availability and reliability, uses EMG and EEG in parallel in a manual control task, measured by 16 EEG channels EEG signals, recording the myoelectric activity of the four channels of right and left forearm flexion and extension.
Based on the classifier output of EMG and EEG, in the first method, equal-balanced fusion weights are used to combine the output of EMG and EEG classifiers; in the second method, Bayesian The fusion method was tested. The effectiveness of the EMG and EEG classifiers used alone was tested, and the four conditions of the fusion of EMG and EMG were considered according to different levels of muscle fatigue.
Figure integrated sensors to drive functional electrical stimulation (FES) based on tremor suppression.
In the experiment, the experiment was carried out on 12 healthy subjects (3 subjects were removed). In this study, the electromyography activity The accuracy rate is 87%, and the accuracy rate of EEG activity is 73%. The accuracy of the first hybrid EMG-EEG classifier fusion method is increased to 91%, and the Bayesian fusion method uses 50% of the EMG signal to achieve 92% accuracy. The experimental results show that it is different from using EEG and EMG input alone. Compared with the situation, the classification accuracy of the joint method is improved.
Tremor is a well-known problem in typical control methods based on electromyographic signals. Tremor is a common disease, especially in the elderly. It causes rhythmic swings of body parts, especially in people with upper limb tremor. Show difficulty in activities of daily living. The EMG signal generated by these unintentional exercises does not represent the user's actual exercise intention. Therefore, it is of great significance to identify and eliminate the chattering effect in the control methods of biological robots such as exoskeleton.
Some studies, such as the use of active wearable exoskeleton to suppress tremor, and avoid unnecessary vibration or movement when using power assisted robots. However, a new multimodal sensor fusion method for suppressing tremor has recently been reported. In this study, we propose a multi-modal BCI-mediated soft wearable robot that can compensate for upper limb tremor by functional electrical stimulation (FES).
In this study, based on the EEG, EMG, and inertial sensor signals, the control signals that drive the fully automatic wearable robot are generated. In this case, the hybrid fusion method used can be classified as a sequence fusion method.
In this method, the first step is to use the surface Laplacian filter of the C3, CZ and C4 electrodes to filter the recorded EEG signals to identify the subject's intentional voluntary movement. Once found, the EMG signal is used to identify the onset of tremor, and finally, the inertial measurement unit (IMU) is used to track the parameters of the tremor.
The picture uses EEG and EMG signals to perform a sensing-assisted upper limb strength assistance
experiment in 12 patients with neurotremor. Using data from the experimental phase for evaluation, the average magnitude estimation error is 0.001 rad/s, and the typical tremor frequency range is The frequency estimate within is used as the final output. In addition to the final results, this study also shows the importance of fusion and integration of different modes to improve the accuracy of detecting and characterizing voluntary and trembling motor components, especially robustness.
Under normal circumstances, the EMG signal is used to judge or verify the perceptual auxiliary function, and the power-assisted exoskeleton robot is taught according to each task. However, in some cases, the EMG signal changes are not enough to make this judgment. In order to overcome this problem, a power-assisted control method for upper limbs combining EEG and EMG signals is proposed. A 256 high-density electrode system is used to measure EEG signals and record the EMG signals of 16 upper limb muscles. The power assist robot performs perception assistance based on information from the surrounding environment (using stereo cameras, ultrasonic sensors, etc.).
The novelty of this research is that in addition to EMG signals, EEG signals are also used to judge the effectiveness of the perceptual assistance performed. Although in this method, the EMG signal and the EEG signal are not directly fused, the EMG signal and the EEG signal are considered at the same time to determine whether the perceptual assistance is performed.
Two experiments were conducted on 4 subjects, and the cognition rate of correct or incorrect using EMG alone and combined with EEG-EMG method was calculated. In the first experiment, the average accuracy of perceptual aid judgments of all subjects were 77.5% and 88.75%, while the EMG alone method and EEG-EMG combined method were 77.5% and 88.75%, respectively; in the second experiment Under the same parameters, the results of the single EMG method and the mixed EEG-EMG method were 57.5% and 80%, respectively. The combination of EEG and EMG signals can improve the judgment of the perceptual assistance effect.
When a specific individual lacks the ability to generate control signals to guide the biorobot device, the EEG-EMG hybrid control method is effective. For example, for amputees above the elbow, the muscles used to generate forearm, wrist, and hand movements do not exist. An additional electrical signal of arm movement (34 degrees) is proposed. The movement of the forearm and wrist is It is estimated by artificial neural network. However, it is not easy to estimate various activities of daily living with this method.
Therefore, in order to control the prosthesis of amputees above the elbow, signals from the brain are proposed. The pronation/supination movement of the forearm of the artificial arm is controlled by the EEG signals measured by a high-density EEG sensor array (256 EEG electrodes), while the elbow flexion and extension movements are controlled by the remaining two. Controlled by the electromyographic signals of the head and triceps. The synchronous processing of the EEG and EMG signal of the amputee above the elbow controlling the prosthesis can be expressed as shown in the figure below.
This method uses an artificial neural network to decode the forearm movement according to the EEG signal, and uses an EMG controller with fuzzy nerve modifier to generate the elbow joint movement of the artificial arm. Compared with previous studies, the hybrid control method based on EEG-EMG shows its potential advantages by introducing additional degrees of freedom (forearm pronation/supination) control channels.
It is interesting to discuss the possibility of integrating other sensors (such as motion sensors or tactile sensors) into the hybrid EEG-EMG method, and report an attempt to design an exoskeleton robot intelligent perception system based on multi-dimensional information fusion. In this research, we propose an exoskeleton robot perception system architecture, using EEG, EMG, plantar pressure sensor and fiber optic motion capture system information fusion, the basic framework of the design is to use EEG The signal is used to determine the direction of the human body's movement intention, and the EMG signal of the four major muscles of the human body's lower limbs is used to identify the human body's movement mode (running or walking). The optical fiber motion capture system is used to measure the position and posture of the human body. Within the framework of the exoskeleton robot perception system, each measurement signal is processed sequentially. In the framework of the design, EEG and EMG signals are not directly combined.
Some research groups have proposed integrating EEG and EMG signals into control methods for applications such as the exoskeleton of biological robots. Bearing in mind the Walker project, researchers have proposed a brain-computer integration method based mainly on EMG and EEG signals related to human movement.
In order to integrate EEG and EMG signals, a design of dynamic recurrent neural network (DRNN) is proposed. This network can receive both the EMG signal of the shoulder muscles that imitate walking, and the spontaneous EEG signals during walking. It is recommended that the comprehensive method of assisted exoskeleton for walking rehabilitation be used to help the disabled with movement disorders. Similar recommendations based on the FES control method of the hybrid BCI system have also been reported for the rehabilitation of upper limbs of patients after stroke. As proposed in this study, in the hybrid method, the EEG pattern is used to recognize the movement intention of a given patient, and only when a specific EMG feature that the patient voluntarily tries is detected, muscle contraction will be produced by FES.
The author concludes that preliminary experiments on healthy volunteers in the laboratory have shown the feasibility of this method, and more tests will be carried out with the participation of stroke patients and rehabilitation experts.
At the same time, the same research team also proposed a BCI-based multi-modal feedback device to support the rehabilitation of upper limbs after stroke. This method uses three 16-channel biosignal amplifiers to measure EEG signals and record the EMG signals of upper limb muscles (digital flexors and extensors, biceps and triceps). The two signals are simultaneously sent to the parallel processing pipeline to process EEG And EMG signal. Only when EEG and EMG appear at the same time (including motor cortex and physiological muscle mode), the fusion module allows FES to be activated. In these experiments, patients can observe that their hands are controlled by the proposed hybrid BCI with the help of FES orthoses.
In addition to the above-mentioned methods, few documents have proposed the conceptual design of hybrid sensor fusion in BCIs. In these designs, not only EEG and EMG signals must be combined, but also other inputs, such as simple switches and motion sensor signals, to improve information. Transmission rate, availability and reliability, etc. Among these methods, the hybrid BCI should determine the input channel according to the user's preference and/or availability, or the combination of fusion channels, to provide the most reliable signal in the assistive technology system.
Although this is not a simple biorobot application, it is interesting that we discussed a study that reported a hybrid control based on a P300 brain-computer interface for communication with severely disabled end users. The main purpose of this study is to evaluate the use of electromyography (EMG) and P300 features from EEG signals to control the hybrid BCI interface of assistive technology software. 8 EEG channels (according to the 10-20 system) and 2 EMG electrodes were used to measure EEG and EMG signals.
Table Reporting Research Comparison of Quantitative Performance Results The
experiment was conducted by six healthy subjects and one end user with severe dyskinesias. During the experiment, participants were asked to use the system to spell three predefined words (21 characters) online in two situations, namely: no mixed task and mixed task.
In non-mixed tasks, this method only uses BCI control, while in mixed tasks it uses EMG control signals to eliminate errors in spelling tasks. Finally, three indicators are used to evaluate the efficiency of the hybrid BCI method: time, error percentage, and user frustration. As shown by the results shown, the efficiency of the hybrid BCI system is higher than that of the non-hybrid version. For healthy subjects, the three measurement results all show that the score of the mixed method is significantly lower than that of the non-mixed method. In addition, compared with the non-hybrid method (time 34.8s and error 33.9%), end users with severe dyskinesia can obtain a lower time average (19.13s) and percentage error (19.3%).
"Shh! Quietly insert a small commercial"
The specific parameters of BCIduino 8-channel EEG amplifier are as follows:
Input impedance: 1TΩ
Input bias current: 300pA
Input reference noise: 1μVpp
Sampling rate: 250 Hz/500Hz
Common mode rejection ratio: -110dB
Adjustable gain amplification factor: 1, 2, 4 , 6, 8, 12
, 24 resolution: 24-bit ADC, accuracy up to 0.1μV
Power consumption: 39mW in normal operation, as low as 10μW in standby,
powered by a rechargeable lithium battery, further reducing external interference.
Size: 50mm*50mm (physical measurement, with slight errors)
Figure 1 BCIduino physical image
Figure 2 BCIduino data waveform in an ordinary noisy environment, suspended state, you can observe no other interference occurs
Figure 3 OpenBCI in an ordinary noisy environment, Data waveform in the floating state (the measurement environment, measurement time, and software filter setting parameters are the same as those in Figure 2BCIduino)
Add management to get more
This article is organized or written by the BCIduino brain-computer interface open source community. The BCIduino brain-computer interface community was initiated and established by masters and doctors from Beihang University, Cornell University, Peking University, Capital Medical University, etc. Welcome to join the community by scanning the code and remark "BCI", and also welcome to purchase BCIduino EEG module (some Search for treasure)