This chapter introduces the LU decomposition method, and how to use the LU decomposition method to find the inverse and determinant, and introduces each formula, principle, and code in detail, hoping to bring you some help.

Table of contents

Determine the L, U matrix

Significance of LU decomposition method

3) Gaussian method inversion

4) Matrix multiplication

LUP finds the determinant

1) Code

2) Code explanation

LU decomposition method

concept

Transform the coefficient matrix A into the product of two equivalent matrices L and U, where L and U are unit lower triangular matrix and upper triangular matrix, respectively. When all the sequential principal subforms of A are not 0, the matrix A can be uniquely decomposed into A=LU. where L is the lower triangular matrix (the main diagonal element is 1), and U is the upper triangular matrix.

$A=LU$

$\begin {bmatrix} a_{11} & a_{12} & a_{13} &... & a_{1n}\\ a_{21} & a_{22} & a_{23} &... & a_{2n }\\ a_{31}& a_{22} & a_{33} &... & a_{3n}\\ ...& ... & ... &... &... \\ a_ {n1}&... &... &... &a_{nn} \end{bmatrix}=\begin{bmatrix}1& & & & \\l_{21}&1 & && \\l_{31}& l_{22} & 1 && \\ ...& ... & ... &1&\\ l_{n1}&... &... &... &1 \end{bmatrix}\begin{bmatrix} u_{11} & u_{12} &u_{13} &... & u_{1n}\\ &u_{22} & u_{23} &... & u_{2n}\\ & & u_{33} &... & u_{3n}\\ & & &... &... \\ & & & &u_{nn} \end{bmatrix}$

Then, inverting the matrix A becomes:

$A^{-1}=\left ( LU \right )^{-1}=U^{-1}L^{-1}$

Because LU is a lower triangular matrix and an upper triangular matrix, the complexity of floating-point operations will be reduced by half when Gaussian transformation is performed to invert the matrix.

Finding the value of its determinant for matrix A becomes:

Because the main diagonal elements of L are all 1, the value of the determinant of A is equal to the product of the main diagonal elements of U.

Determine the L, U matrix

Because the main diagonal elements of the matrix L are fixed to 1, the elements of the matrix L and U can be deduced row by row and column by column by the principle of matrix multiplication

Methods as below:

Determine the first row of U
Determine the first column of L
Determine the second row of U
Determine the second column of L
...
Determine the nth row of U
Determine the nth column of L

Determine the first row of U :

The first row of L is multiplied and added to the corresponding elements of the first column of U, and the result is equal to $a_{11}$ , that is: $1\cdot u_{11}=a_{11}$
The first row of L is multiplied and added to the corresponding elements of the second column of U, and the result is equal to $a_{12}$ , that is: $1\cdot u_{12}=a_{12}$
It is not difficult to see that the first row of U has the same elements as the first row of A: $u_{1j}=a_{1j}$

Determine the first column of L:

The first element of each column of L is 1, and the judgment starts from the second element
The second row of L is multiplied and added to the corresponding elements of the first column of U, and the result is equal to $a_{21}$ , that is: $l_{21}\cdot u_{11}=a_{21}$
The third row of L is multiplied and added to the corresponding elements of the first column of U, and the result is equal to $a_{31}$ , namely: $l_{31}\cdot u_{11}=a_{31}$
Therefore, the first column of L is: $l_{n1}=\frac{a_{n1}}{u_{11}}=\frac{a_{n1}}{a_{11}}$

Following the above steps, L and U can be obtained, and the finishing formula is as follows:

$u_{ij}=a_{ij}- \sum_{k=1}^{i-1}l_{ik}u_{ki}(i=2,...,n;j=i,...,n)$

$l_{ij}={\left ( a_{ij}-\sum_{k=1}^{j-1}l_{ik}u_{ki} \right )}/{u_{jj}} (j=1,2,...,n-1;i=j+1,...n)$

When designing the code, the summation part can be calculated separately with the for loop and stored in the variable a. The variable a needs to be initialized to 0. When i=0, the condition of the for loop is not satisfied. If a=0, the above U can be deduced. First row and first column of L.

For the derivation and verification of the LU decomposition method, please refer to the relevant numerical analysis textbooks. It is recommended to refer to the direct LU decomposition method of the blog matrix

Significance of LU decomposition method

LU is inherited, which is the advantage of LU.

LU is only suitable for solving all sequential main sub-expressions greater than 0, lack of versatility, which is the shortcoming of LU.

The LU method is not guaranteed to be numerically stable, which is a disadvantage of LU. (The Gauss method can use the column pivot technique to ensure numerical stability)

Combining the advantages of LU and Gauss while avoiding these disadvantages is the LUP decomposition method.

Author: Zhai Sen Lambda-CDM

My personal understanding is: when a computer processes a hyperdimensional matrix (for example, 10,000 dimensions), it will use a block matrix to solve the problem. By first block and then LU, the matrix is divided into a block of lower triangular matrix and upper triangular matrix $L_{n}$ storage $U_{n}$ . In general, in subsequent other calculations, it is more efficient to directly call the calculation of the lower triangular matrix and the upper triangular matrix. (I am also learning this part, welcome to discuss)

programming

This part of the program is actually the LUP decomposition method, which adds the process of selecting the pivot, because when decomposing LU and Gaussian elimination for inversion, if the element of the pivot is 0, the computer will not be able to calculate, so before decomposition, you need to first Select the pivot.

LUP inversion

1) Code

#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<malloc.h>

double** Matrix_LU_Inv(double** src)
{
	if (src == NULL)exit(-1);
	int row = (int)_msize(src) / (int)sizeof(src[0]);
	int col = (int)_msize(*src) / (int)sizeof(src[0][0]);
	if (row != col)exit(-1);
	int i, j,k,max;
	double** L, ** U, ** tmp,**Linv,**Uinv,**res,Lsum, Usum,p;
	L = (double**)malloc(sizeof(double*) * row);
	U = (double**)malloc(sizeof(double*) * row);
	tmp = (double**)malloc(sizeof(double*) * row);
	if (L && U)
	{
		for (i = 0; i < row; i++)
		{
			L[i] = (double*)malloc(sizeof(double) * col);
			U[i] = (double*)malloc(sizeof(double) * col);
			tmp[i] = (double*)malloc(sizeof(double) * col);
			memset(L[i], 0, sizeof(L[0][0]) * col);//L U需初始化为0
			memset(U[i], 0, sizeof(U[0][0]) * col);
			memcpy(tmp[i], src[i], sizeof(src[0][0]) * col);//拷贝数据
		}
	}
	for (i = 0; i < row; i++)
	{
		for (j = 0; j < row; j++)
		{
			if(i==j)
			L[i][j] = 1;//L主对角线为1
		}
	}
	//选主元
	for (j = 0; j < col; j++)
	{
		max = j;
		double Max = fabs(tmp[max][j]);//用绝对值比较
		//默认当前主元最大
		for (i = j; i < row; i++)
		{
			if (fabs(tmp[i][j]) > Max)
			{
				max = i;
				Max = fabs(tmp[i][j]);
			}
		}
		if (i == j && i != max)
		{
			for (k = 0; k < col; k++)
			{
				p = tmp[max][k];
				tmp[max][k] = tmp[i][k];//交换两行
				tmp[i][k] = p;
			}
		}
	}
	for (i = 0; i < row; i++)
	{
		for (j = 0; j < col; j++)
		{
			if (i <= j)
			{
				Usum = 0;
				for (k = 0; k < i ; k++)
				{
					Usum += L[i][k] * U[k][j];//计算求和部分
				}
				U[i][j] = tmp[i][j] - Usum;
			}
			else
			{
				Lsum = 0;
				for (k = 0; k < j ; k++)
				{
					Lsum += L[i][k] * U[k][j];//计算求和部分
				}
				L[i][j] = (tmp[i][j] - Lsum) / U[j][j];
			}
		}
	}
	Linv = Matrix_inver(L);//求逆
	Uinv = Matrix_inver(U);
	res = Matrix_Mul(Uinv, Linv);//乘法
	free(L);free(Linv);free(U);free(Uinv);free(tmp);
	return res;
}

2) Code explanation

Line 3: Determine whether the pointer to the incoming matrix is empty

Lines 4~6: Judge the dimension of the matrix. _msize is a function in <malloc.h>, which returns the size of the memory pointed to by the address. This part is described in detail in C Language Matrix Dimension Judgment

Line 7: i, j, k are the variables used for the loop, and max is the number of rows where the maximum number of records is located when selecting the pivot

Line 8: L, U correspond to the LU matrix; tmp is the temporary matrix created to protect the original matrix; Linv, Uinv correspond to its inverse matrix; res is the final output inverse matrix; Lsum and Usum correspond to the summation part in the formula; p is a temporary variable for exchanging two rows of elements

Lines 9~23: open up memory space for the above matrix, initialize L and U to 0, and copy the content of the original matrix to tmp

Lines 24~31: Elementize the main diagonal of L to 1

Lines 33~55: Select the pivot, the default is the largest current pivot, select the pivot from below the main diagonal (protect the selected pivot).

Line 60: Because when calculating LU, the row of U is calculated first, and then the column of L is calculated alternately; only need to judge the size of row number i and column number j: when i <= j, it needs to be calculated If the element is above the main diagonal, U is calculated; otherwise, L is calculated below the main diagonal (because the main diagonal of L is 1, there is no need to calculate, and the main diagonal can be used as the dividing line)

Lines 65 and 74: This part corresponds to the previous formula. Before each recalculation, Usum and Lsum need to be reset to 0

Lines 80 and 81: the matrix inversion function created by Matrix_inver (arr), the matrix inversion function can refer to the previous blog:

C language matrix inversion (Gaussian method) and C language matrix inversion (adjoint method) , the time complexity of the adjoint method is too high, the Gaussian method is recommended (see the code below) .

Line 82: The matrix multiplication function created by Matrix_Mul (A, B) (see the code below) , which multiplies A from the left of B. Refer to the blog: Matrix multiplication in C language

Line 83: Pay attention to freeing memory!

test:

3) Gaussian method inversion

#include<stdio.h>
#include<stdlib.h>
#include<math.h>
#include<string.h>
 
double** Matrix_inver(double** src)
{
	//step 1
	//判断指针是否为空
	if (src == NULL)exit(-1);
	int i, j, k, row, col, n;
	double** res, ** res2,tmp;//res为增广矩阵，res为输出的逆矩阵
	int count = 0;
	//判断矩阵维数
	row = (double)_msize(src) / (double)sizeof(double*);
	col = (double)_msize(*src) / (double)sizeof(double);
	if (row != col)exit(-1);
	//step 2
	res = (double**)malloc(sizeof(double*) * row);
	res2 = (double**)malloc(sizeof(double*) * row);
	n = 2 * row;
	for (i = 0; i < row; i++)
	{
		res[i] = (double*)malloc(sizeof(double) * n);
		res2[i] = (double*)malloc(sizeof(double) * col);
		memset(res[i], 0, sizeof(res[0][0]) * n);//初始化
		memset(res2[i], 0, sizeof(res2[0][0]) * col);
	}
	//step 3
	//进行数据拷贝
	for (i = 0; i < row; i++)
	{
		memcpy(res[i], src[i], sizeof(res[0][0]) * n);
	}
	//将增广矩阵右侧变为单位阵
	for (i = 0; i < row; i++)
	{
		for (j = col; j < n; j++)
		{
			if (i == (j - row))
				res[i][j] = 1.0;
		}
	}
	
	for (j = 0; j < col; j++)
	{
       //step 4
	   //整理增广矩阵，选主元
		count = j;
		double Max = fabs(res[count][j]);//用绝对值比较
		//默认第一行的数最大
		//主元只选主对角线下方
		for (i = j; i < row; i++)
		{
			if (fabs(res[i][j]) > Max)
			{
				count = i;
				Max = fabs(res[i][j]);
			}
		}
		if (i == j && i != count)
		{
			for (k = 0; k < n; k++)
			{
				tmp = res[count][k];
				res[count][k] = res[i][k];
				res[i][k] = tmp;
			}
		}
        //step 5
		//将每列其他元素化0
		for (i = 0; i < row; i++)
		{
			if (i == j || res[i][j] == 0)continue;
			double b = res[i][j] / res[j][j];
			for (k = 0; k < n; k++)
			{
				res[i][k] += b * res[j][k] * (-1);
			}
		}
		//阶梯处化成1
		double a = 1.0 / res[j][j];
		for (i = 0; i < n; i++)
		{
			res[j][i] *= a;
		}
	}
	//step 6
	//将逆矩阵部分拷贝到res2中
	for (i = 0; i < row; i++)
	{
		memcpy(res2[i], res[i] + row, sizeof(res[0][0]) * row);
	}
	//必须释放res内存！
	free(res);
	return res2;
}

4) Matrix multiplication

#include<stdio.h>
#include<stdlib.h>

double** Matrix_Mul(double** arr1, double** arr2)
{
	if (arr1 == NULL || arr2 == NULL)exit(-1);
	int row1 = (int)_msize(arr1) / (int)sizeof(double*);
	int col1 = (int)_msize(*arr1) / (int)sizeof(double);
	int row2 = (int)_msize(arr2) / (int)sizeof(double*);
	int col2 = (int)_msize(*arr2) / (int)sizeof(double);
	if (col1 != row2)
		exit(-1);//判断左列是否等于右行
	double** res = (double**)malloc(sizeof(double*) * row1);
	if (res == NULL)
		exit(-1);
	int i, j, k;
	for (i = 0; i < row1; i++)
	{
		res[i] = (double*)malloc(sizeof(double) * col2);//创建新矩阵
	}
	for (i = 0; i < row1; i++)
	{
		for (j = 0; j < col2; j++)
		{
			res[i][j] = 0.0;//开辟的新矩阵未初始化，计算前需要进行初始化
			for (k = 0; k < col1; k++)
			{
				res[i][j] += arr1[i][k] * arr2[k][j];//该部分的计算与前文一致
			}
		}
	}
	return res;
}

LUP finds the determinant

1) Code

double Matrix_LU_Det(double** src)
{
	if (src == NULL)exit(-1);
	int row = (int)_msize(src) / (int)sizeof(src[0]);
	int col = (int)_msize(*src) / (int)sizeof(src[0][0]);
	if (row != col)exit(-1);
	int i, j, k, max;
	double** L, ** U, ** tmp, Lsum, Usum, p,res=1;
	L = (double**)malloc(sizeof(double*) * row);
	U = (double**)malloc(sizeof(double*) * row);
	tmp = (double**)malloc(sizeof(double*) * row);
	if (L && U)
	{
		for (i = 0; i < row; i++)
		{
			L[i] = (double*)malloc(sizeof(double) * col);
			U[i] = (double*)malloc(sizeof(double) * col);
			tmp[i] = (double*)malloc(sizeof(double) * col);
			memset(L[i], 0, sizeof(L[0][0]) * col);//L U需初始化为0
			memset(U[i], 0, sizeof(U[0][0]) * col);
			memcpy(tmp[i], src[i], sizeof(src[0][0]) * col);//拷贝数据
		}
	}
	for (i = 0; i < row; i++)
	{
		for (j = 0; j < row; j++)
		{
			if (i == j)
				L[i][j] = 1;//L主对角线为1
		}
	}
	//选主元
	for (j = 0; j < col; j++)
	{
		max = j;
		double Max = fabs(tmp[max][j]);//用绝对值比较
		//默认第一行的数最大
		for (i = j; i < row; i++)
		{
			if (fabs(tmp[i][j]) > Max)
			{
				max = i;
				Max = fabs(tmp[i][j]);
			}
		}
		if (i == j && i != max)
		{
			for (k = 0; k < col; k++)
			{
				p = tmp[max][k];
				tmp[max][k] = tmp[i][k];
				tmp[i][k] = p;
			}
		}
	}
	//计算L、U
	for (i = 0; i < row; i++)
	{
		for (j = 0; j < col; j++)
		{
			if (i <= j)
			{
				Usum = 0;
				for (k = 0; k < i; k++)
				{
					Usum += L[i][k] * U[k][j];
				}
				U[i][j] = tmp[i][j] - Usum;
			}
			else
			{
				Lsum = 0;
				for (k = 0; k < j; k++)
				{
					Lsum += L[i][k] * U[k][j];
				}
				L[i][j] = (tmp[i][j] - Lsum) / U[j][j];
			}
		}
	}
	for (i = 0; i < row; i++)
	{
		for (j = 0; j < col; j++)
		{
			if (i == j)
				res *= U[i][j];
		}
	}
	free(L);  free(U);  free(tmp);
	return res;
}

2) Code explanation

LUP is relatively simple to find the determinant, there is no additional function

Note: The pivot selection in LUP does not need to record the number of row transformations, but Gaussian elimination needs to record the number of row transformations in the determinant

(Gaussian elimination to find the determinant, see C language to find the determinant (Gaussian method) )

After decomposing LU, you only need to multiply the main diagonal elements of U, and res needs to be 1 during initialization.

test:

C language - use the matrix LU decomposition method to find the inverse and determinant

LU decomposition method

concept

Determine the L, U matrix

Significance of LU decomposition method

programming

LUP inversion

1) Code

2) Code explanation

3) Gaussian method inversion

4) Matrix multiplication

LUP finds the determinant

1) Code

2) Code explanation

Guess you like