[Action Pattern Recognition] Realize compound action pattern recognition (offline control module)

1. Ideas

Generally speaking, to realize the face touching action, it is necessary to collect the electromyographic signals of the following muscles of the arm:

• Biceps brachii: controls elbow flexion
• Triceps brachii: controls extension of the elbow joint
• Forearm flexor muscles: control flexion of the wrist and fingers
• Forearm extensor muscles: control extension of the wrist and fingers
• Abductor thumb longus: controls the expansion of the thumb
• Extensor pollicis brevis: controls extension of the thumb

The specific location of these muscles can be found at: http://menovomed.com/newsv.php?id=549. Generally speaking, electrodes need to be attached to the target muscles, taking care to avoid cross interference and noise.
(1) Arm movement recognition and virtual simulation based on myoelectric signals - Baidu Academic. https://xueshu.baidu.com/usercenter/paper/show?paperid=10550cv0wc3104x00v1h0vr0sp742042.
(2) With thoughts as "commanders", prostheses can also Can be very "obedient"----Chinese Academy of Sciences. https://www.cas.cn/kx/kpwz/201708/t20170821_4611750.shtml.
(3) Arm movement pattern recognition based on surface electromyographic signals - topic opening PPT_Baidu Library. https://wenku.baidu.com/view/4e6fff73a417866fb84a8e4c.html.
(4) Deep learning human action recognition algorithm - Zhihu - Zhihu column. https://zhuanlan.zhihu.com/p/46599638.
(5) Ultra-lightweight and more general! Action recognition based on human skeleton points - Zhihu. https://zhuanlan.zhihu.com/p/426695879.
(6) With the mind as the "commander", the prosthetic limbs can also be "obedient"----Chinese Academy of Sciences. https ://bing.com/search?q=%e6%89%8b%e8%87%82%e5%8a%a8%e4%bd%9c%e6%a8%a1%e5%bc%8f%e8% af%86%e5%88%ab.

2. How to process signals for complex actions?

If you collect five channels of electromyographic signals at one time, you can add these signals together, that is, add the signals of each channel to get a total signal, and then perform preprocessing, feature extraction and patterning on this total signal. Identify. The advantage of this is that it can reduce the amount of calculation and improve the signal-to-noise ratio. However, this method also has some disadvantages, such as the possibility of losing some information between channels, or causing the signals of some channels to be masked by the signals of other channels.

Another method is to perform preprocessing and feature extraction on the signals of each channel separately, and then splice the feature vectors of each channel into a total feature vector, and then use this total feature vector for pattern recognition. The advantage of this is that it can retain the information of each channel, or exploit the correlation between different channels. However, this method also has some disadvantages, such as possibly increasing the amount of calculation and reducing the signal-to-noise ratio.

Therefore, you need to choose the appropriate method based on your specific application scenarios and goals.
Recommend a very good domestic tutorial from Station B : https://www.bilibili.com/video/BV15e4y197iL/?spm_id_from=333.999.0.0&vd_source=e1b57954a90a5c44920c0ea78136af67

(1) Summary of basic knowledge of surface electromyography signal (sEMG) - Zhihu - Zhihu column. https://zhuanlan.zhihu.com/p/485705730.
(2) Dry information | Detailed explanation of sEMG surface electromyography in one article - Zhihu - Zhihu column. https://zhuanlan.zhihu.com/p/379925002.
(3) What is myoelectricity? _What does electromyography mean - CSDN Blog. https://blog.csdn.net/psybrain/article/details/120945604.
(4) Research on multi-channel surface electromyography signal decomposition - Baidu Academic. https://xueshu. baidu.com/usercenter/paper/show?paperid=96753679fcfd2908ec256deced56da2b.
(5) NinaPro-DB1 · master · mirrors / malele4th / sEMG_DeepLearning. https://gitcode.net/mirrors/malele4th/sEMG_DeepLearning/-/tree/master/NinaPro -DB1.
(6) Dry information | Detailed explanation of sEMG surface electromyography in one article - Zhihu - Zhihu column. https://zhuanlan.zhihu.com/p/379925002.

3. Feature extraction ideas

First, let’s take a look at what feature extraction algorithms are suitable for myoelectric signals?
There are many types of feature extraction algorithms for electromyographic signals, and different algorithms may be suitable for different purposes and scenarios. Generally speaking, feature extraction algorithms for electromyographic signals can be divided into the following categories:

• Time domain-based feature extraction algorithm: This type of algorithm mainly uses the change pattern of electromyographic signals in time to extract some statistics or timing parameters, such as mean, variance, standard deviation, skewness, kurtosis, peak-to-trough distance, Zero crossing rate etc. The advantage of this type of algorithm is that it is simple and easy to implement and has a small amount of calculation, but the disadvantage is that it is sensitive to noise and cannot reflect the frequency characteristics of the electromyographic signal.
• Feature extraction algorithm based on frequency domain: This type of algorithm mainly uses the distribution characteristics of electromyographic signals in frequency to extract some spectral domain parameters or transformation coefficients, such as power spectral density, spectrum center, spectrum width, spectrum fundamental frequency, spectrum maximum Peak values, fast Fourier transform coefficients, wavelet transform coefficients, etc.¹². The advantage of this type of algorithm is that it can reflect the frequency characteristics of the electromyographic signal and is insensitive to noise, but the disadvantage is that it requires a large amount of calculation and requires a high sampling rate.
• Feature extraction algorithm based on time-frequency domain: This type of algorithm mainly uses the joint characteristics of electromyographic signals in the time domain and frequency domain to extract some time-frequency parameters or transformation coefficients, such as short-time Fourier transform coefficients and continuous wavelet transform coefficients, discrete wavelet transform coefficients, empirical mode decomposition coefficients, etc. The advantage of this type of algorithm is that it can reflect the time variability and nonlinearity of electromyographic signals and is suitable for non-stationary signals. However, the disadvantage is that it requires greater calculation and requires higher sampling rate and resolution.
  We choose the continuous wavelet transform coefficient algorithm, which is not sensitive to noise and needs to meet a high sampling rate
• Feature extraction algorithm based on deep learning: This type of algorithm mainly uses deep neural networks to automatically learn high-level abstract features of electromyographic signals without manual design or selection of features, such as convolutional neural networks, recurrent neural networks, autoencoders, etc. The advantage of this type of algorithm is that it can adapt to complex and diverse electromyographic signal data and improve recognition accuracy and robustness. However, the disadvantage is that it requires a large amount of training data and computing resources, and it is difficult to interpret and understand the meaning of features.
  For composite actions, it is more appropriate to choose an algorithm based on the time-frequency domain. Why?

For compound actions, such as touching the face, touching the back of the head, eating, etc., I think the feature extraction algorithm based on the time-frequency domain is more suitable for your requirements than the feature extraction algorithm based on the frequency domain. Because the feature extraction algorithm based on the time-frequency domain can reflect the time domain and frequency domain characteristics of the electromyographic signal at the same time, while the feature extraction algorithm based on the frequency domain can only reflect the frequency domain characteristics of the electromyographic signal. Compound movements usually involve the coordination and changes of multiple muscle groups, so the time variability and nonlinearity of the electromyographic signal need to be considered, not just the frequency characteristics.

There are many feature extraction algorithms based on time-frequency domain, such as short-time Fourier transform (STFT), continuous wavelet transform (CWT), discrete wavelet transform (DWT), empirical mode decomposition (EMD), etc. You can choose the appropriate algorithm and parameters based on your data and goals. Generally speaking, STFT and CWT can provide higher time resolution and frequency resolution, but require greater calculation and storage space; DWT can provide lower time resolution and frequency resolution, but require smaller Computational amount and storage space; EMD can provide adaptive time resolution and frequency resolution, but requires higher computational complexity and stability.

About training set and test set
Training set and test set are two types of data sets commonly used in machine learning. Their meanings and differences are as follows:

• The training set is the data used to train the model. It contains input variables and output variables, that is, features and labels. The labels in the training set can be manually labeled or obtained from other data sources, such as databases, websites, sensors, etc.
• The test set is data used to evaluate the performance of the model. It also contains input variables and output variables, but will not be used during the training process. The labels in the test set can also be manually labeled or obtained from other data sources, but they must be consistent with the label source of the training set to ensure the validity of the evaluation.

There are many ways to divide the training set and the test set. One common method is the holdout cross validation method , that is, the data set is statically divided into the training set and the test set according to a certain ratio, such as 6:4 or 8:2. Another common method is k-fold cross validation , which dynamically divides the data set into k parts. Each time, one part of the k parts is used as the test set, and the others are used as the training set, repeated k times. , and then take the average as the final evaluation result.

To learn more about the definitions and differences between training sets and test sets, you can refer to the following web pages

(1) Machine learning essentials: training set, verification set and test set - Zhihu. https://zhuanlan.zhihu.com/p/377789735.
(2) Understanding AI data sets in one article: training set, verification set, Test set (attached: segmentation method + cross-validation) - Zhihu. https://zhuanlan.zhihu.com/p/98532085.
(3) What do the test set and training set mean? - Baidu Knows. https://zhidao.baidu.com/question/425949299.html.
(4) Why does machine learning need to be divided into training set, test set and verification set? What is the difference between these three sample sets? - Zhihu. https://zhuanlan.zhihu.com/p/377396096.
(5) # Popular Understanding# The difference between test set, training set and development set - Zhihu. https://zhuanlan.zhihu.com/p /367995480.

4. Action pattern recognition

When processing three-channel EMG signals or more channel signal data for action pattern recognition, it is crucial to choose an appropriate algorithm. Depending on different needs, several algorithms may be suitable:

1. Linear Discriminant Analysis (LDA) : LDA is a simple and computationally efficient method suitable for situations where the feature space is linearly separable. It performs classification by maximizing inter-class distance and minimizing intra-class distance.

2. Support Vector Machine (SVM) : SVM performs well in processing small samples, nonlinear and high-dimensional pattern recognition. It distinguishes different categories of data by finding the optimal hyperplane.

3. Artificial Neural Network (ANN) : ANN is capable of learning and simulating nonlinear and complex relationships, especially when the feature space is complex or when the data relationship is difficult to describe using traditional statistical methods.

4. Convolutional Neural Network (CNN) : CNN performs well in image processing. For electromyographic signals, it can be converted into electromyographic images, and then CNN can be used for feature learning and classification.

5. Recurrent Neural Network (RNN) : RNN is particularly suitable for processing sequence data and can capture dynamic changes in time series. It is suitable for time-varying data such as electromyographic signals.

Which algorithm to choose depends on your specific application scenario and data characteristics. For example, if your data set is small, SVM may be a good choice; if you need to deal with complex patterns or classification boundaries, ANN or CNN may be more suitable. Meanwhile, deep learning methods such as CNN and RNN can automatically extract features and reduce the need for manual feature engineering, but they usually require more data and computing resources.

When choosing an algorithm, you should also consider real-time requirements and the computing power of your system. For example, LDA may be more suitable for applications requiring fast response due to its simplicity. Deep learning methods, despite their high computational complexity, can provide higher accuracy and robustness when sufficient resources are available.

In short, choosing an appropriate algorithm requires comprehensive consideration of data characteristics, application requirements, computing resources, and real-time requirements.

(1) Research progress on electromyographic pattern recognition in intelligent interaction - SIAT. http://neural.siat.ac.cn/index.php/news/109.shtml.
(2) Action recognition based on surface electromyographic signals - Deep learning method - Zhihu. https://zhuanlan.zhihu.com/p/61650163.
(3) Research on perception and prediction of human lower limb movement intention based on EMG - Docin.com. https://www.docin.com/ p-2321503571.html.
(4) Multi-stream convolutional network fusion gesture recognition method of surface electromyographic signals. http://www.shcas.net/jsjyup/pdf/2022/8/Multi-stream convolution of surface electromyographic signals Accumulative network fusion gesture recognition method.pdf.
(5) New trend of "inertial motion capture + wireless surface electromyography" technology integration - Zhihu. https://zhuanlan.zhihu.com/p/411813910.
(6) Intelligent interaction Research progress on electromyographic pattern recognition: https://mp.weixin.qq.com/s/QkOxEseGpQX-KhMvap9SzA.