[Entropy and feature extraction] From approximate entropy, to sample entropy, to fuzzy entropy, and then to permutation entropy, what has been achieved? (Part Four) - "Permutation Entropy" and its MATLAB Implementation

Today I talk about permutation entropy. I used three articles to describe approximate entropy, sample entropy and fuzzy entropy:

Mr. Kanhai: [Entropy and Feature Extraction] From approximate entropy, to sample entropy, to fuzzy entropy, and then to permutation entropy, what has been achieved? (Part 1) - "Approximate Entropy" and its MATLAB Implementation

Mr. Kanhai: [Entropy and Feature Extraction] From approximate entropy, to sample entropy, to fuzzy entropy, and then to permutation entropy, what has been achieved? (Part Two) - "Sample Entropy" and its MATLAB Implementation

Mr. Kanhai: [Entropy and Feature Extraction] From approximate entropy, to sample entropy, to fuzzy entropy, and then to permutation entropy, what has been achieved? (Part Three) - "Fuzzy Entropy" and its MATLAB Implementation

Permutation entropy is quite different from the first three methods in principle, so the theoretical part must abandon inertial thinking to a certain extent and accept new algorithm concepts. Let's start the last one in this entropy series (maybe).

1. Permutation entropy

Permutation Entropy (Permutation Entropy, PE) is a method proposed by Bandt and Pompe [1] to detect the randomness and dynamic mutation behavior of time series. It has the advantages of simple and fast calculation, strong anti-noise ability, and suitable for online monitoring . It has been widely used in EMG signal and heart rate signal analysis, temperature complexity and mechanical fault detection.

1.1 The meaning of permutation entropy

Suppose the signal sequence to be analyzed is x(1), x(2),...,x(N).

(1) [Sequence Definition] Given a pattern dimension m, a set of m-dimensional vectors can be constructed: $X(i)=[x(i),x(i+\year),...x(i+\year(m-1))]$

It should be noted that the above sequence has a parameter $\ can$ called time delay, and its value must be a positive integer. In fact, this parameter can be understood as the downsampling of the sequence. For example, every 3 data points were sampled $= 3$ at that time . $\tau = 1$ At that time, the sequence was defined the same as the sequence of approximate entropy.

In fact, the concept of time delay is also introduced in approximate entropy, sample entropy, and fuzzy entropy, but it is usually $\ can$ set to 1.

(2) [Sequence sorting] Sort the elements in the X(i) sequence, for example, when the elements in X(i) are {20,60,10}, the sorting can be written as (2,3,1), The numbers in the sorting represent the order of each element. (There is also another way of understanding. The sorting will be written as (3,1,2), which means that the order is the number of the original sequence. These two understandings do not affect the final calculation result).

(3) [Probability of sorting] For an m-dimensional vector, there are m possible arrangements! kind. Define the probability of occurrence of each sort as follows: $P(i)=\frac{Num(X(i))}{N-(m-1)\lambda}$

(4) [Permutation Entropy] Calculate permutation entropy according to the definition of information entropy: $H_{PE}(m)=-\sum_{i=1}^{N-(m-1)\lambda}{P(i)log_{2}P(i)}$

Note that I specifically marked the base as 2 when calculating the logarithm above, because some papers here use e as the base, and the method proposer just wrote log without marking the base, but the author verified the original paper. In this case, the base number used by the method proponent is 2. However, after normalization in the next step, it doesn't really matter what the base number is.

(5) [Normalization] In order to adjust the range of entropy value to the range of 0 to 1, normalize the data: $H_{NPE}(m) = \frac{H_{PE}(m)}{log_{2}(m!)}$

1.2 Characteristics of Permutation Entropy

Permutation entropy has the advantages of simple algorithm, fast calculation speed, strong anti-noise ability, etc. It is widely used in mechanical fault diagnosis, biological signal processing and other fields.

1.3 Parameter selection of permutation entropy

The calculation of permutation entropy mainly involves m and $\ can$ these two parameters.

(1) The choice of m. Different from m in approximate entropy, sample entropy and fuzzy entropy, which usually takes 1 or 2, the recommended value of m in permutation entropy is 3~7. When the embedding dimension m is too small, the probability patterns of the reconstructed sequence are less. When the embedding dimension m is too large, the calculation time will increase and the calculation efficiency will decrease. In other words, if it is acceptable for the program to run for a long time, m can be set as large as possible.

(2) $\ can$ Choice. If the amount of signal data is not particularly large, there is no need to downsample the signal, usually set to 1. Of course, there are also in-depth studies on the influence of phase space reconstruction on different values. In this case, the value needs to be set according to the situation.

2. MATLAB code implementation

The code of permutation entropy can be found in the third-party version on the Internet, and the official library of MATLAB has not yet included it. The author modified and commented on the reliable version, made some corrections according to the above theory, and added comments line by line, and explained it in comparison with the algorithm steps in the above article. I named it kPermutationEntropy as follows:

For the ease of use of the feature extraction code, the author encapsulates a series of entropy feature extractions, including the codes added with comments above. Since when doing scientific research and writing papers, the needs for feature extraction are often centralized, multi-category, and different in demand, so I packaged the entropy feature values introduced before and the centralized entropy features that will be reduced later. :

There are 7 entropy eigenvalues - power spectrum entropy, singular spectrum entropy, energy entropy, approximate entropy, sample entropy, permutation entropy , fuzzy entropy

The above 7 types are all concentrated into one package function to realize feature extraction with one line of code.

For example, to extract data "permutation entropy" can be written like this:

fea = genFeatureEn(data,{'PeEn'}) %对data求排列熵

If all seven features of the data " power spectral entropy, singular spectral entropy, energy entropy, approximate entropy, sample entropy, permutation entropy, and fuzzy entropy " are extracted, it can be written as follows:

fea =genFeatureEn(data,{'psdE','svdpE','eE', 'ApEn', 'SpEn','PeEn','FuzzyEn'});  
%调用genFeature函数，完成特征提取，算出的特征值会保存在fea变量里

That is to say, which feature needs to be extracted can be directly specified in the function. The corresponding eigenvalues will be obtained in the output fea variable, and the order is also consistent with the order of the input.

The introduction of this function is as follows:

function fea = genFeatureEn(data,featureNamesCell,options)
% 熵相关算法的信号特征提取函数
% 输入：
% data：待特征提取的时域信号，可以是二维数据，维度为m*n，其中m为数据组数，n为每组数据的长度。即每行数据为一组。行列方向不可出错
% options：其他设置，使用结构体的方式导入。目前可设置变量包括：
%   -svdpEn：即奇异值的窗口长度。
%   -Apdim：近似熵参数，Apdim为近似熵的模式维度
%   -Apr：近似熵参数，Apr为近似熵的阈值
%   -Spdim：样本熵参数，Spdim为样本熵的模式维度
%   -Spr：Spr为样本熵的阈值
%   -Fuzdim：模糊熵参数，Fuzdim为模糊熵模式维度
%   -Fuzr：模糊熵参数，Fuzr为模糊熵的阈值
%   -Fuzn：模糊熵参数，Fuzn为模糊熵权重
%   -Pedim：排列熵参数，Pedim为排列熵模式维度
%   -Pet：排列熵参数，Pet为排列熵的时间延迟
% featureNamesCell：拟进行特征提取的特征名称，该变量为cell类型，其中包含的特征名称为字符串，特征名称需要在下边列表中：
% 目前支持的特征（2022.5.23，共7种）：
%      psdE：功率谱熵
%      svdpE：奇异谱熵
%      eE：能量熵
%      ApEn：近似熵
%      SampleEn：样本熵
%      FuzzyEn：模糊熵
%      PerEn：排列熵
% 
% 输出：
% fea：数据data的特征值数组，其特征值顺序与featureNamesCell一一对应
%

Students who need the above function file and test code can get it at the following link:

Feature Extraction (Entropy Feature Metrics) | Toolbox Documentation

3. Others: MATLAB code implementation of feature extraction in time domain and frequency domain

In addition to the extraction of the above-mentioned entropy features, the author also coded the time-domain and frequency-domain features mentioned in the previous article, including:

There are 8 dimensional eigenvalues—maximum value, minimum value, peak-to-peak value, mean value, variance, standard deviation, mean square value, root mean square value (RMS) and 6 dimensionless eigenvalues—kurtosis, skewness, Five frequency-domain eigenvalues of shape factor, crest factor, pulse factor, and margin factor—centroid frequency, mean square frequency, root mean square frequency, frequency variance, frequency standard deviation 4 spectral kurtosis characteristics—spectral kurtosis mean, standard deviation of spectral kurtosis, skewness of spectral kurtosis, kurtosis of spectral kurtosis

The above 23 types are all concentrated into one package function to realize feature extraction with one line of code.

For example, extracting the data "center of gravity frequency" can be written like this:

fea = genFeatureTF(data,{'FC'}) %对data数据求重心频率

If you extract all 22 features of the data "maximum value, minimum value, peak-to-peak value, mean value, variance, standard deviation, mean square value...", you can write like this:

fea =genFeatureTF(data,{'max','min','mean','peak','arv','var','std','kurtosis',...
               'skewness','rms','waveformF','peakF','impulseF','clearanceF',...
               'FC','MSF','RMSF','VF','RVF',...
               'SKMean','SKStd','SKSkewness','SKKurtosis'});  %调用genFeature函数，完成特征提取，算出的特征值会保存在fea变量里

The introduction of this function is as follows:

function fea = genFeatureTF(data,fs,featureNamesCell)                                           
% 时域、频域相关算法的信号特征提取函数
% 输入：
% data：待特征提取的时域信号，可以是二维数据，维度为m*n，其中m为数据组数，n为每组数据的长度。即每行数据为一组。行列方向不可出错
% fs：采样频率，如果不提取频域特征，fs值可以设置为1
% featureNamesCell：拟进行特征提取的特征名称，该变量为cell类型，其中包含的特征名称为字符串，特征名称需要在下边列表中：
% 目前支持的特征（2022.5.23，共23种）：
% max ：最大值
% min ：最小值
% mean ：平均值
% peak ：峰峰值
% arv  ：整流平均值
% var  ：方差
% std  ：标准差
% kurtosis  ：峭度
% skewness  ：偏度
% rms       ：均方根
% waveformF ：波形因子
% peakF     ：峰值因子
% impulseF  ：脉冲因子
% clearanceF：裕度因子
% FC：重心频率
% MSF：均方频率
% RMSF：均方根频率
% VF：频率方差
% RVF：频率标准差
% SKMean：谱峭度的均值
% SKStd：谱峭度的标准差
% SKSkewness：谱峭度的偏度
% SKKurtosis：谱峭度的峭度
% 
% 输出：
% fea：数据data的特征值数组，其特征值顺序与featureNamesCell一一对应

Students who need the above function file and test code can get it at the following link:

Feature extraction (time domain, frequency domain indicators) | Toolbox Documentation

The above two functions (the entropy feature extraction function "genFeatureEn" and the time-frequency feature extraction function "genFeatureTF") will be continuously updated. If there are any feature indicators that you want to add, students can leave a message in the comment area, and the author will consider including it. In "Feature Extraction Index Family Bucket".

refer to

^[1] Bandt C , Pompe B . Permutation Entropy: A Natural Complexity Measure for Time Series[J]. Physical Review Letters, 2002, 88(17):174102.