"Numerical Analysis" - 1 - Solving Equations

0. Background

Solving equations is one of the most important problems in engineering calculations. This chapter focuses on $f(x)=0$ different methods for determining the solution x of an equation.

Why do I need to know more than one method of solving equations? The choice of method depends on $f$ the cost required to evaluate the function or its derivative. Different usage scenarios have different convergence speed and computational complexity requirements for the solution method.

forward error and reverse error

The approximate solution we $f(x)=0$ get for solving the equation is $x_a$ , and the exact solution of the equation is $r$

forward error: $\left | r-x_a \right |$

Backward error: $\left | f(x_a) \right |$

Backward error, measured vertically, is usually much smaller than forward error measured horizontally.

The termination criteria for the equation solving method can be based on forward or backward errors.

sensitive issue

If a small error in the input, in which case the problem is solved, causes a large problem in the output, such a problem is called a sensitivity problem. Suppose a small change is made to the input $\varepsilon g(x)$

Sensitive formula for root: $\triangle r=-\frac{\varepsilon g(r) }{f'(r)}=-\frac{g(r) }{f'(r)}\varepsilon$

$\varepsilon$ The previous coefficient is the error amplification factor, which is defined as {relative forward error}/{relative backward error}

1. Dichotomy

The main idea

According to the median value theorem, first roughly determine the location of the root, judge whether it is located in the left half or the right half of the existing interval, and then further subdivide the interval where the root is located.

specific algorithm

Assuming that the initial interval is $[a,b]$ , after n times of bisection, the resulting interval is $[a_n,b_n]$ of length $(b-a)/2^n$ , and the midpoint is chosen $x_c=(a_n+b_n)/2$ as the best estimate of the value.

theorem

If the error is less than $0.5\cdot 10^{-p}$ , the solution is accurate to one decimal $p$ place.

features

In the process of binary iteration, it is not necessary to know the entire curve of the function, only to calculate the value of the function at the necessary position.

For the dichotomy method, once the expected accuracy is determined, the necessary number of iterations can be calculated through theorem.

2. Fixed point iterative FPI

definition

When $g(r)=r$ , the real numbers $r$ are $g$ the fixed points of the function.

The main idea

All equations $f(x)=0$ can be transformed into a fixed- $f(x)=x$ point iterative problem, and the same equation can have multiple transformation forms, some transformation forms can converge to the fixed point, and some forms cannot converge. It is essentially a linear convergence and divergence problem of errors.

specific algorithm

First $f(x)=0$ rewrite it into a suitable $g(x)=x$ form;

given $x_0$ = initial estimate;

execute $x_{i+1}=g(x_i)$ ;

until $\left | x_{i+1}-x_i \right |<TOL$

theorem

Assuming the function $g$ is continuously differentiable, $g(r)=r$ , $S=\left | g'(r) \right |<1$ , then the fixed point iteration converges to the fixed point linearly with velocity for a sufficiently close $r$ initial estimate . $S$ $r$

features

The dichotomy can guarantee linear convergence, the fixed point iteration is only local convergence, and when the fixed point iteration converges, it converges linearly. The dichotomy removes 1/2 of the uncertainty at each step, while in FPI the uncertainty is multiplied at each step $S=\left | g'(r) \right |$ , so the fixed point iteration may be faster or slower than the dichotomy, depending on S in 1/2 size relationship.

Moreover, the fixed point iteration method does not guarantee that it will converge to $r$ .

3. Newton's method

Newton's method is an improved algorithm of FPI, also known as Newton-Raphson method.

The main idea

From $x_0$ the beginning, draw the tangent to the function , and mark the intersection with $y=f(x)$ the x-axis as the next approximation to the root of the function, and repeat the whole process. $x_1$

specific algorithm

$x_0$ = initial value;

$x_{i+1}=x_{i}-\frac{f(x_i)}{f'(x_i)}$

...

theorem

Assuming that the function $f$ is a second-order continuous differentiable function, $f(r)=0$ , if $f'(r)\neq 0$ , then Newton’s method converges locally quadratically $r$ , and $i$ the error of the first step $e_i$ satisfies:

$lim_{t\rightarrow \infty } \frac{e_{i+1}}{e_i^2}=M$

in,

$M=\frac{f''(r)}{2f'(r)}$

features

Newton's method does not always converge quadratically, it converges linearly at multiple root locations, where the coefficient of linear convergence $S=(m-1)/m$ .

where m multiple roots are defined as:

If $0=f(r)=f'(r)=...=f^{(m-1)}(r)$ , but $f^{(m)}(r)=0$ .

4. Secant method and its variants

The reason why the Newton method can achieve a faster iteration speed is because more information is used, especially the information of the tangent direction of the function obtained through the function derivative. But in some cases, it may be difficult to compute the derivative, in which case the secant method is a replacement for Newton's method, which uses the approximate secant instead of the tangent, and converges about as fast.

The specific process of secant method

$x_0,x_1$ = initial estimate

$x_{i+1}=x_{i}-\frac{f(x_i)(x_i-x_{i-1})}{f(x_i)-f(x_{i-1})}$

The difference quotient is used instead of the derivative, so the method requires two initial estimates.

The secant method has three generalized forms: trial position method, Muller method, and inverse quadratic difference method IQI.

Specific process of trial position method

Given a range $[a,b]$ such that $f(a)(b)<0$

$for\, i=1,2,3...$

         $c=\frac{bf(a)-af(b)}{f(a)-f(b)}$

         $if f(c)=0,stop,end$

$if f(a)f(c)<0$

$b=c$

   $else$

                 $a=c$

         $end$

$end$

It can be seen that the main idea of the test position method is to continuously update the root-containing interval. It starts to perform better than the dichotomy method and the secant method, and has the best properties of the two, but it cannot guarantee that it can be eliminated at every step like the dichotomy method. 1/2 uncertainty.

Muller method and inverse quadratic difference method IQI

These two methods both use parabolas to replace straight lines (both require three initial estimates for parabolic interpolation). The difference is that the parabola used by the Muller method has multiple $y=p(x)$ intersections with the x-axis, while the inverse quadratic difference method IQI The parabolic form adopted $x=p(y)$ will only have one intersection point with the x-axis.

5. Brent method

The Brent method is a hybrid method. The original intention is to combine the guaranteed convergence properties of the bisection method with the fast convergence properties of more complex methods, and the Brent method does not require derivative calculations.

"Numerical Analysis" - 1 - Solving Equations

0. Background

1. Dichotomy

2. Fixed point iterative FPI

3. Newton's method

4. Secant method and its variants

5. Brent method

Guess you like