Matlab nonlinear iterative method (3) damped Newton method LM

Detailed explanation of Gauss Newton method_I'm just an automatic frog's blog-CSDN Blog

1. Thought

Let's first look at the disadvantages of the Newton-Gauss iterative method:

1. In the process of calculation, there may be a singular matrix (unsatisfactory rank), for example: J(k)​)TJ(k) cannot get the correct solution when it is an ill-conditioned matrix, or when seeking inv((J(k )​)TJ(k) ) is irreversible, then this cannot be calculated anymore​

2. When Δxk is too large, it may cause x(k+1)=x(k)+Δxk The iteration is not accurate and the algorithm does not converge

Therefore: On this basis, the confidence interval u is proposed

 For ill-conditioned matrices, LM proposes a method of using coefficient matrix damping to transform the matrix J(k)​)TJ(k) so that the algorithm can be calculated.

Two, steps

1. Initial value setting u and coefficient initial value a b cd 

2. Calculate (Jf'*Jf +uI)delta_x=Jf'f(x) to solve delta_x,

3. Judgment accuracy

4、xk=xk+delta_x

5. Judge p and iterate out new u

p>0.75  u=2u  

p<0.25  u=0.5u

When ρ is close to 1, the approximation effect is good;
when ρ is too small, the actual reduction value is much smaller than the reduction value of the approximation function, and the approximation effect is poor, and the approximation range needs to be reduced. 
When ρ is large, the actual reduction value is greater than The value of the approximation function decreases, the approximation effect is poor, and the approximation range needs to be increased 
 

% L-M 迭代算法 
close all;
clear ;
clc;
% y(i)=a*exp(-(x(i)-b).^2/c.^2)+0.1*rand(1); 函数模型
% 在拟合数据的时候用的系数是
% a=1;
% b=4;
% c=10;
% 因此在拟合参数的时候，这里用 a=0.866
a=0.966;
b=3.98;
c=9.98;
d=0;
% 设置优化半斤
u=0.001;

segma=0.0000000001;% 精度
iterator_num=100;% 最大跌代次数
x=load('x.mat');
act_x=x.x;
y=load('y.mat');
len=length(act_x);
jacobian_d=ones(len,1);
I=ones(4,4);
act_y=y.y;
plot(act_x,act_y,'.');
hold on;
for i=1:iterator_num
  % 计算误差
    y=a*exp(-(act_x-b).^2/c.^2)+d;
    r=act_y-y;
    % 开始计算偏导数矩阵
    jacobian_a=exp(-(act_x - b).^2/c.^2);
    jacobian_b=(a*exp(-(act_x - b).^2/c.^2).*(2.*act_x - 2*b))/c.^2;
    jacobian_c=(2*a*exp(-(act_x - b).^2/c.^2).*(act_x - b).^2)/c.^3;
    % jacobian_d
    Jf=[jacobian_a,jacobian_b,jacobian_c,jacobian_d];
    gx=Jf'*r;
    %(Jf'*Jf  +uI)delta_abcd=Jf'f(x)
    % H =Jf'*Jf
    H=Jf'*Jf;
    delta_abcd=inv(Jf'*Jf+u.*I)*Jf'*r;
    %delta_abcd=inv(Jf'*Jf)*Jf'*r;
    %Jf'*Jf+u
    % g=H*delta_abc  增量方程
    if norm(delta_abcd)<segma
        break;
    end
    
    % 计算 pa pb pc pd 
    a_iterator=a+delta_abcd(1);
    b_iterator=b+delta_abcd(2);
    c_iterator=c+delta_abcd(3);
    d_iterator=d+delta_abcd(4);
    u=correction_p(act_x,delta_abcd,u,y,a_iterator,b_iterator,c_iterator,d_iterator,Jf);
    
    a=a_iterator;
    b=b_iterator;
    c=c_iterator;
    d=d_iterator;
end
it_y=a*exp(-(act_x-b).^2/c.^2)+d;
plot(act_x,it_y,'-');
legend('act','fit','Location','southoutside','Orientation','horizontal')

% 计算相关性
function  u=correction_p(act_x,delta,u,y,a,b,c,d,Jf)
 y_1=a*exp(-(act_x-b).^2/c.^2)+d;
 pp=Jf*delta;
 ys=y_1-y;
 p=ys\pp;
 if p>0.75
   x=2*u;
 elseif p<0.25
    x=0.5*u;
 end
 u=x;
end 

 

 

Please point out the mistakes and discuss together! ! !