[Mathematics] Popular understanding of Taylor's formula (Newton iteration method is useful)

[Mathematics] Popular understanding of Taylor's formula (Newton iteration method is useful)

1 Introduction

Recently, I was looking at some methods related to machine learning optimization (gradient descent, Newton iteration, etc.), which also involved Taylor formula expansion, etc. I almost forgot what I learned in college, so I read some blogs and sorted them out.

Taylor formula, also known as Taylor expansion. It is a formula that uses the information of a function at a certain point to describe its nearby values. If the function is smooth enough, when the derivative values ​​of each order of the function at a certain point are known, the Taylor formula can use these derivative values ​​as coefficients to construct a polynomial approximation function to obtain the value in the neighborhood of this point.

• So what is the Taylor formula for?
  • To put it simply, it is to use a polynomial function to approximate a given function (that is, to make the polynomial function image fit the given function image as much as possible). Note that when approximating, it must be expanded from a certain point on the function image.
  • If you want to find a certain value of a very complex function, but it is impossible to find it directly, you can use Taylor's formula to approximate the value, which is one of the applications of Taylor's formula.
  • Taylor's formula is mainly used in gradient iteration (Newton iteration method) in machine learning.

2. Popular understanding

The following formula is the simplest class of elementary functions, namely polynomials.

• The operation of polynomial itself is only addition, subtraction and multiplication of finite items, so in terms of numerical calculation, polynomial is a tool that people are willing to use. Therefore, we often use polynomials to approximate expression functions. This is why Taylor's formula chooses polynomial functions to approximate a given function.

2.1 Approximate calculation

Elementary mathematics has learned some functions such as:

but elementary mathematics has not answered how to calculate them, $f(x)=\cos The\; approximate\; calculation\; of\; x$ is taken as an example, here is the calculation:

1) One-time (linear) approximation

Use differential approximation formula $f(x)\approx f(x_0)+f^{\prime }(x_0)(x-x_0)$ (this formula is converted from the limit expression formula of derivative/differentiation), for$x_0=$f ( x ) f(x)around$0$$The\; linear\; approximation\; of\; f\left(x\right)$ is:$f\left(x\right)\approx f(0)+f^{\prime }(0)x$ , 所以 $f\left(x\right)=\cos x\approx 1$ , so$f\left(x\right)$ at$x_0=$Linear approximation function around$0$$P_1(x)=1$ , as shown in the figure below:

• Linear approximation-advantages: simple form, convenient calculation;
• Linear Approximation - Disadvantage: The further you are from the origin O, the worse the approximation.

2) Quadratic approximation

Using the quadratic polynomial $p_2\left(x\right)=a_0+a_{1}x+a_2{x}^2$ to approximate$f(x)=\cos x$ , we expect:

所以 $\cos x\approx P_2\left(x\right)=1-\frac{x^2}{2}$,As shown below:

• Quadratic approximation is much better than linear approximation, but limited to $[-\frac{}{2}，\frac{}{2}]$ , outside of this range, the images are significantly different.
• Why do we expect the function value, first derivative value, and second derivative value of two functions to be equal at a certain point?
  • Because these values ​​express the most basic and main properties of the function (image), the approximation of these properties can make the two functions approximate (it can be seen intuitively from the above function image). This is also the basic idea of ​​Taylor's formula.

3) Eight approximations

Using an octagonal polynomial $p_8(x)=a_0+a_{1}x+a_2{x}^2+...+a_8{x}^8$ to approximate$f(x)=\cos x$ , we expect:

Therefore, we get:

• The image is as follows:
  

• $P_8\left(x\right)$ (green image) than$P_2\left(x\right)$ (blue image) is closer to the cosine function in a larger range (red image)

In summary, it can be seen from the above three different degrees of function approximation: when the accuracy requirement is high and the error needs to be estimated, a high-degree polynomial must be used to approximate the expression function, and the error formula is given at the same time. The above is a process of using polynomial functions to approximate a given function.

3. Derivation of Taylor's formula

This leads to a question: Given a function $f\left(x\right)$ , to find a point at the specified point$x_0$Nearby with $f\left(x\right)$ very approximate polynomial function$P\left(x\right)$ , written as:

making$f\left(x\right)\approx P_n\left(x\right)$ and make both errors$R_n\left(x\right)=f\left(x\right)-P_n\left(x\right)$ can be estimated. So what conditions should the polynomial to find meet, and what is the error?

• Geometrically, $y=f\left(x\right)$，$y=P_n\left(x\right)$ represents two curves, as shown in the figure below:

make them at $x_0$The vicinity of is very close, it is obvious:

1. Firstly, two curves are required to be in $\left(x_0,f\left(x_0\right)\right)$ point intersection, that is,$P_n\left(x_0\right)=f\left(x_0\right)$

2. If you want to get closer, it is also required that the two curves are at $\left(x_0,f\left(x_0\right)\right)$ points are tangent, (It can be seen intuitively from the image that the intersection [brown and red image] and tangent [green and red image], the two curves are at$x_0$The proximity of the vicinity is obviously different, and the tangency is closer), that is, ${P_n}^{\prime }\left(x_0\right)=f^{\prime }\left(x_0\right)$

3. If you want to get closer, the curve is also required to be at $\left(x_0,f\left(x_0\right)\right)$ points have the same bending direction, (as shown in the figure above, the bending direction is opposite [green and red images]; the bending direction is the same [blue and red images], obviously at a distance from$x_0$Far away, the difference between the two functions in the same bending direction is smaller), that is, ${P_n}^{\prime \prime }\left(x_0\right)=f^{\prime \prime }\left(x_0\right)$ , and then it can be deduced: if at$\left(x_0,f\left(x_0\right)\right)$ near${P_n}^{\prime }\left(x_0\right)=f^{\prime }\left(x_0\right)$，${P_n}^{\prime \prime }\left(x_0\right)=f^{\prime \prime }\left(x_0\right)\cdots \cdots \cdots P_n^{\left(n\right)}\left(x_0\right)=f^n\left(x_0\right)$ , the approximation gets better and better.

To sum up, the polynomial to be found should satisfy the following conditions:

• Explain how the above conversion is done, taking the second derivative of the third line above as an example:
  • Transformation of the first arrow: put $P_n\left(x\right)$ After calculating the second order derivative function,$x_0$Bring in, get ${P_n}^{\prime \prime }\left(x_0\right)=2!a_2$
  • Transformation of the second arrow: so $f^{\prime \prime }\left(x_0\right)=2!a_2$，所以 $a_2=\frac{1}{2!}{f}^{\prime \prime }\left(x_0\right)$

polynomial function $p_n(x)=a_0+a_1(x-x0)+a_2(x-x0{)}^2+...+a_n(x-x0{)}^{}$Coefficient a \small ain${}^n$$a$ can be all composed of$f\left(x\right)$ , then get:

where the error is$R_n\left(x\right)=f\left(x\right)-P_n\left(x\right)$ . Because polynomial functions are used to infinitely approximate a given function, there must be a slight error between the two.

4. Definition of Taylor formula

So we get the definition of Taylor's formula:

If the function $f\left(x\right)$ containing$x_0$An open interval of $\left(a,b\right)$ with up to$\left(n+1\right)$ order derivative, then for$\forall x\in \left(a,b\right)$ , there are:

the remainder (ie error)$R_n\left(x\right)=\frac{f^{\left(n+1\right)}(\xi }{\left(n+1\right)!}(x-x_0{)}^{n+1}$ \$\xi$ at$x_0$with $between\; x$ . There are several ways to express the remainder of Taylor's formula. The previous expression is called the Lagrangian remainder of n-th order Taylor expansion.

• The Lagrangian remainder is the nth-order Taylor formula expanded by one more order, and n becomes n+1.
• Note that the residual term here is the error, because using a polynomial function to expand at a certain point and approximate a given function will definitely have a little bit of error in the end, which we call the residual term.

5. Extension - McLaughlin formula

McLaughlin's formula is a special case of Taylor's formula: when $x_0=$Taylor's formula at$0\; .$So$x_0=Substituting\; 0$ into the formula, we get:

McLaughlin's formula with Peano remainders for several common elementary functions:

Peano remainders are${\left(x-x_0\right)}^{}$Higher-order infinitesimals of${}^{n\; :}$

reference

【1】https://blog.csdn.net/xiaojinger_123/article/details/127442655