"Numerical Analysis" - 2-equation system

0. Background

In the previous chapter, we studied methods for solving univariate equations. In this chapter, the simultaneous solution of many multivariate equations will be studied, with a great deal of attention devoted to problems with as many unknown variables as there are equations.

Gaussian elimination is a useful tool for solving linear equations of appropriate size.

1. Gaussian Elimination

Gaussian elimination can efficiently solve n equations with n unknowns.

Gaussian elimination is mainly composed of two unequal parts, the elimination process with relatively large computational cost, and the back-substitution process with relatively small computational cost.

The elimination calculation of n equations and n unknowns can $\frac{2}{3}n^3+\frac{1}{2}n^2-\frac{7}{6}n$ be completed after one operation.

2. LU decomposition

LU decomposition is the matrix form of Gaussian elimination, which involves writing the coefficient matrix A as the product of the lower triangular matrix L and the upper triangular matrix U. If the problem solved by Gaussian elimination is Ax=b, then after LU decomposition, the problem is transformed into LUx=b.

Among them, the U matrix is the upper triangular matrix obtained by the traditional Gaussian elimination process, and the corresponding L matrix is obtained by placing 1 on the main diagonal, and then placing the multipliers in the lower triangular matrix according to their specific positions during elimination.

How to use LU decomposition to convert back to the generation step?

Ax=b ---> LUx=b --->c=Ux Lc=b find c---> Ux=c find x

Since both L and U are triangular matrices, the operation of these two steps is very straightforward.

Why do LU decomposition?

Generally, the problem of solving Ax=b we encounter is often a set of different problems. Usually the matrix A is constant, but b changes frequently. Therefore, b is isolated from the calculation including A through LU decomposition, and only one elimination process is performed. can solve such problems.

However, not all matrices can be decomposed by LU, we need to do some work before LU decomposition: some pivots.

3. Error

Infinity norm

$x=(x_1,x_2,...,x_n)$ The infinite norm or maximum norm of the vector is ${\left | \left | x \right | \right |}_\infty=max \left | x_i \right |$ , that is, the maximum absolute value among all elements of x.

Let $x_a$ be the approximate solution to the system of linear equations $Ax=b$ , and the remainder be a vector $r=b-Ax_a$ . The backward error is the norm of the remainder ${\left \| b-Ax_a \right \|}_\infty$ , the forward error is ${\left \| x-x_a \right \|}_\infty$ , the relative backward error is $\frac{{\left \| r \right \|}_\infty }{{\left \| b \right \|}_\infty }$ , and the relative forward error $\frac{{\left \| x-x_a \right \|}_\infty }{{\left \| x \right \|}_\infty }$

The error amplification factor is:

$\frac{\frac{{\left \| x -x_a \right \|}_\infty }{{\left \| x \right \|}_\infty }}{\frac{{\left \| r \right \|}_\infty }{{\left \| b \right \|}_\infty }}$

condition number of the matrix

The condition number cond(A) of matrix A is the maximum possible error amplification factor for all right vector b when solving Ax=b

matrix norm

The matrix norm of the nxn matrix A is ${\left \| A \right \|}_\infty$ = the maximum value of the sum of the absolute values of the elements in each row, that is, the absolute value of the elements in each row is summed, and the maximum value of the sum of n rows is used as the norm of the matrix A.

Surprisingly, there is a compact formula for the condition number for square matrices:

$cond(A)=\left \| A \right \|\left \| A^{-1} \right \|$

1-norm of matrix

The matrix 1-norm of an nxn matrix A is ${\left \| A \right \|}_1$ = maximum absolute column sum, i.e. the maximum value of the 1-norm of the column vectors.

4.PA=LU decomposition

The Gaussian elimination method we considered earlier is called a "naive" problem because of two serious obstacles: 0 pivot (which makes elimination impossible) and the drowning problem (too large multipliers cause the bottom equation to be suppressed). For a non-singular matrix, the two can be avoided by improving the algorithm. The improved strategy focuses on exchanging the rows of the coefficient matrix, a method called partial pivoting.

partial pivot

The idea of partial pivot is to find the largest element in the first column before each elimination step, and exchange its corresponding row with the pivot row, that is, PA=LU. Among them, P is the final accumulated permutation matrix for pivot selection at each step.

Using the partial pivot method, the problem of Ax=b is further transformed into PAx=Pb -->LUx=Pb

Then get c from Lc=Pb, and get x from Ux=c.

The advantage of the Gaussian pivot method is that it has a process of selecting the pivot, which can avoid the situation that the program selects the principal element as 0 when performing the elimination operation, and also reduces the rounding error of the calculation, thereby improving the program. generalizability and accuracy of results.

5. Iterative method

The Gaussian method is a straightforward method for solving systems of linear equations. Theoretically, direct methods can obtain exact solutions in finite steps.

Iterative methods can also be used to solve linear systems of equations. Similar to fixed-point iteration, the method starts with an initial estimate, then refines the estimate at each step, and finally converges to the solution vector.

Jacobian method

The Jacobian method is a form of fixed-point iteration in systems of equations.

The Jacobian method proceeds in the standard manner as follows:

Solve the i-th equation to get the i-th unknown quantity;

Then use fixed point iteration, starting from the initial estimate, to iterate.

But the Jacobian method doesn't always succeed in solving it, and knowing the conditions under which it works helps our understanding.

First, let's understand the concept of a strictly diagonally dominant matrix.

Definition of Strictly Diagonally Dominant Matrix

The diagonal element is dominant in the corresponding row, and its corresponding value (absolute value) is greater in number than the sum of the other elements in the row.

Properties of Strictly Diagonally Dominant Matrices

If the nxn matrix A is strictly diagonally dominant, then A:

1) is a non-singular matrix;

2) For all vectors b and initial estimates, applying the Jacobian method to Ax=b converges to a unique solution.

Jacobian Concrete Algorithm

Matrix A can be written as A=L+D+U, and the solution equation can be written as Lx+Dx+Ux=b, which can be further transformed into $x=D^{-1}(b-(L+U)x_k)$

Then the Jacobian iterative process is:

$x_0$ = initial vector;

$x_{k+1}=D^{-1}(b-(L+U)x_k)$

Gauss-Seidel method

The difference between the Gauss-Seidel method and the Jacobian method is that the value of the unknown variable updated by the Gauss-Seidel method is always updated at each step, even if the update occurs at the current step.

The iterative process of the Gauss-Seidel method is:

$x_0$ = initial vector;

$x_{k+1}=D^{-1}(b-Ux_k-Ux_{k+1})$

Sequential over-relaxation method (SOR)

The continuous over-relaxation method uses the solution direction of the Gauss-Seidel method and uses over-relaxation to speed up the convergence.

SOR can be seen as multiplying Ax=b by w and reorganizing the equation:

$(wL+wD+wU)x=wb$

$(wL+D)x=wb-wUx+(1-w)Dx$

$x=(wL+D)^{-1}[(1-w)Dx-wUx]+w(wL+D)^{-1}b$

"Numerical Analysis" - 2-equation system

0. Background

1. Gaussian Elimination

2. LU decomposition

3. Error

4.PA=LU decomposition

5. Iterative method

Guess you like