Kernel Extreme Learning Machine (KELM) Classification Algorithm Based on Seagull Algorithm Optimization
Article directory
Abstract: This paper uses the seagull algorithm to optimize the Kernel Extreme Learning Machine (KELM) and use it for classification
1. Theoretical basis of KELM
Kernel Based Extreme Learning Machine (KELM) is an improved algorithm based on Extreme Learning Machine (ELM) combined with kernel functions. KELM can improve the predictive performance of the model while retaining the advantages of ELM.
ELM is a single hidden layer feed-forward neural network, and its learning objective function F(x) can be expressed as a matrix: F ( x
) = h ( x ) × β = H × β = L (9) F(x) =h(x)\times\beta=H\times\beta=L \tag{9}F(x)=h(x)×b=H×b=L( 9 )
where:xxx is the input vector,h ( x ) h(x)h(x)、 H H H is the hidden layer node output,β ββ is the output weight,LLL is the desired output.
Turn network training into a linear system solution problem, β \betaβ according toβ = H ∗ ⋅ L β=H * ·Lb=H∗⋅L is determined, where,H ∗ H^*H∗ isHHThe generalized inverse matrix of H. In order to enhance the stability of the neural network, the regularization coefficientCCC and Identity MatrixIII , then the least squares solution of the output weight is
β = HT ( HHT + I c ) − 1 L (10) \beta = H^T(HH^T+\frac{I}{c})^{-1 }L\tag{10}b=HT(HHT+cI)−1L( 10 )
Introduce the kernel function into ELM, the kernel matrix is:
Ω ELM = HHT = h ( xi ) h ( xj ) = K ( xi , xj ) (11) \Omega_{ELM}=HH^T=h(x_i )h(x_j)=K(x_i,x_j)\tag{11}OhELM=HHT=h(xi)h(xj)=K(xi,xj)( 11 )
In formula:xi x_ixi ， x j x_j xjAs the input vector for the test, formula (9) can be expressed as:
F ( x ) = [ K ( x , x 1 ) ; . . . ; K ( x , xn ) ] ( IC + Ω ELM ) − 1 L ( 12) F(x)=[K(x,x_1);...;K(x,x_n)](\frac{I}{C}+\Omega_{ELM})^{-1}L \tag {12}F(x)=[K(x,x1);...;K(x,xn)](CI+OhELM)−1L( 12 )
where:( x 1 , x 2 , … , xn ) (x_1 , x_2 , …, x_n )(x1,x2,…,xn) for a given training sample,nnn is the sample size.K ( ) K()K ( ) is the kernel function.
2. Classification problem
In this paper, we classify breast tumor data. The training set and test set are generated by random method, in which the training set contains 500 samples, and the test set contains 69 samples.
3. KELM based on seagull algorithm optimization
The specific principle of the seagull algorithm refers to the blog: https://blog.csdn.net/u011835903/article/details/107535864
It can be seen from the above that this paper uses the Seagull algorithm to optimize the regularization coefficient C and the kernel function parameter S. The fitness function is designed as the error rate of the training set and the test set.
fitness = argmin(T rain E rror R ate + T est E rror R ate ). fitness = argmin(TrainErrorRate + TestErrorRate).fitness=argmin(TrainErrorRate+TestErrorRate)。
4. Test results
Correct rate of SOA-KELM in training set: 1
Correct rate of SOA-KELM in test set: 0.95652 Total
number of cases: 569 Benign: 357 Malignant: 212 Total number of cases in
training set: 500 Benign: 300 Malignant: 200 Total
number of cases in test set: 69 Benign: 57 Malignant : 12
benign breast tumor diagnosis: 55 misdiagnosis: 2 diagnosis rate p1=96.4912%
malignant breast tumor diagnosis: 11 misdiagnosis: 1 diagnosis rate p2=91.6667%
training set KELM correct rate: 1
test set KELM correct rate: 0.89855
total number of cases: 569 Benign: 357 Malignant: 212
Total number of cases in training set: 500 Benign: 300 Malignant: 200 Total
number of cases in test set: 69 Benign: 57 Malignant: 12
Diagnosis of benign breast tumors: 55 Misdiagnosis: 2 Diagnosis rate p1=96.4912%
Diagnosis of malignant breast tumors: 7 Misdiagnosis: 5 Confirmed rate p2=58.3333%
It can be seen from the results that Seagull-KELM is significantly better than the original KELM algorithm