Taylor formula (Taylor expansion) popular introduction + detailed explanation of its essence

Let’s briefly explain what Taylor’s formula is.

Taylor formula, also called Taylor expansion. It is a formula that uses information about a function at a certain point to describe its nearby values. If the function is smooth enough, and the derivative values of each order of the function at a certain point are known, Taylor's formula can use these derivative values as coefficients to construct a polynomial approximation function to find the value in the neighborhood of this point.

So what does Taylor's formula do?

Simply put, it is to use a polynomial function to approximate a given function (that is, try to fit the polynomial function image to the given function image). Note that the approximation must be expanded from a certain point on the function image. If you want to find the value of a certain point of a very complex function, but it cannot be achieved directly, you can use Taylor's formula to approximate the value. This is one of the applications of Taylor's formula. Taylor's formula is mainly used in gradient iteration in machine learning.

***********************************************************************************************************************************

1. Raising the question

Polynomials are the simplest class of elementary functions. Regarding polynomials, since its own operations are only addition, subtraction and multiplication of finite terms, polynomials are a tool that people are happy to use in terms of numerical calculations. Therefore we often use polynomials to approximate functions. This is why Taylor's formula chooses a polynomial function to approximate a given function.

***********************************************************************************************************************************

2. Examples of approximate calculations

Elementary mathematics has learned some important properties of some functions such as:, but elementary mathematics has not answered how to calculate them. Take the $\small \cos x$ approximate calculation of f(x) = as an example:

①. First order (linear) approximation

Using the differential approximate calculation formula f(x) $\small \approx$ f( $\small x_{0}$ ) + ${f}'$ ( $\small x_{0}$ )(x - $\small x_{0}$ ) (this formula is converted from the derivative/differential limit expression formula), the $\small x_{0}$ linear approximation of f(x) near = 0 is: f( x) $\small \approx$ f(0) + ${f}'$ (0) x , so f(x) = 1, so the linear approximation function ( $\small \cos x$ $\small \approx$ x) = 1 of f( x) near = 0, as shown below: $\small x_{0}$ $P_{1}$

Advantages of linear approximation: simple form and convenient calculation; disadvantage: the further away from the origin O, the worse the approximation.

②. Quadratic approximation

Quadratic polynomial approximation f(x) = $\small \cos x$ , we expect:

$\small P_{2}\left ( 0 \right )$ = $\small f\left ( 0 \right )$ = $\small \cos 0$ = 1 = $\small a_{0}$ (that is, it is expected that the function values of the approximating function and the given function at x = 0 are equal);

$\small {P_{2}}'\left ( 0 \right )$ = $\small f{}'\left ( 0 \right )$ = $\small \sin 0$ = 0 = $\small a_{1}$ (that is, it is expected that the slope of the approximating function at x = 0 is equal to the slope of the given function);

$\small {P_{2}}''\left ( 0 \right )$ = $\small {f}''\left ( 0 \right )$ = $\small -\cos 0$ = -1, so $\small a_{2}$ = $\small -\frac{1}{2}$ (that is, it is expected that the curvature of the approximating function at x = 0 is equal to the curvature of the given function);

So $\small \cos x$ $\small \approx$ $\small P_{2}\left ( x \right )$ = 1 - $\small \frac{x^{2}}{2}$ , as shown below:

The quadratic approximation is much better than the linear approximation, but it is limited to [ $\small -\frac{\pi }{2}$ , $\small \frac{\pi }{2}$ ], and outside this range, the images are obviously very different. Why do we expect that the function values, first derivative values, and second derivative values of two functions at a certain point are equal? Because these values express the most basic and main properties of the function (image), the approximation of these properties can make the two functions approximate (it can be seen intuitively from the function image above)

③. Eight times of approximation

Using an octave polynomial approximation f(x) = $\small \cos x$ , we expect:

$\small P_{8}\left (0 \right ) = f\left ( 0 \right )$ , find out $\small a_{0} = 1$ (that is, it is expected that the function value of the approximating function at x = 0 is equal to the function value of the given function);

$\small {P_{8}}'\left ( 0 \right ) = {f\left ( 0 \right )}'$ , find out $\small a_{1} = 0$ (that is, it is expected that the slope of the approximating function at x = 0 is equal to the slope of the given function);

.... .... ....

$\small {P_{8}}^{(8)}\left ( 0 \right ) = f^{(8)}(0)$ , find out $\small a_{8} = \frac{1}{8!}$ (that is, it is expected that the curvature of the approximating function at x = 0 is equal to the curvature of the given function);

So , as shown below:

$\small P_{8}\left ( x \right )$ (green image) is $\small P_{2}\left ( x \right )$ closer to the cosine function (red image) over a larger range than (blue image)

It can be seen from the above three different degrees of function approximation: when the accuracy requirements are high and the error needs to be estimated, high-order polynomials must be used to approximate the expression function, and the error formula must be given.

The above is a process of using polynomial functions to approximate a given function.

*****************************************************************************************************************************************

3. Derivation of Taylor’s formula

This leads to a question: given a function $\small f\left ( x \right )$ , find a polynomial function that $\small x_{0}$ is close to the specified point near the specified point , which is recorded as: $\small f\left ( x \right )$ $\small P\left ( x \right )$

Make $\small f\left ( x \right )$ $\small \approx$ $\small P_{n}\left ( x \right )$ and make both errors $\small R_{n}\left ( x \right ) = f\left ( x \right ) - P_{n}\left ( x \right )$ estimable. So what conditions should the polynomial you are looking for satisfy, and what is the error?

Geometrically, $\small y = f\left ( x \right )$ , $\small y = P_{n}\left ( x \right )$ represents two curves, as shown below:

Keeping them $\small x_{0}$ close together makes it obvious:

1. First, the two curves are required $\small \left ( x_{0},f\left ( x_{0} \right ) \right )$ to intersect at a point , that is $\small P_{n}\left ( x_{0} \right ) = f\left ( x_{0} \right )$

2. If you want to get closer, it is also required that the two curves are tangent at $\small \left ( x_{0},f\left ( x_{0} \right ) \right )$ a point . (It can be seen intuitively from the image that the intersection [brown and red image] and tangency [green and red image], the two curves are $\small x_{0}$ close to each other nearby. Obviously the difference is very large, and the tangency is closer), that is $\small {P_{n}}'\left ( x_{0} \right ) = {f}'\left ( x_{0} \right )$

3. If you want to get closer, you also need the curves $\small \left ( x_{0},f\left ( x_{0} \right ) \right )$ to bend in the same direction at points (as shown in the figure above, the bending directions are opposite [green and red images]; the bending directions are the same [blue and red images], obviously at points $\small x_{0}$ far away place, the difference between the two functions with the same bending direction is smaller), that is $\small {P_{n}}''\left ( x_{0} \right ) = {f}''\left ( x_{0} \right )$ , it can be further inferred: if $\small \left ( x_{0},f\left ( x_{0} \right ) \right )$ there is , nearby $\small {P_{n}}'\left ( x_{0} \right ) = {f}'\left ( x_{0} \right )$ , $\small {P_{n}}''\left ( x_{0} \right ) = {f}''\left ( x_{0} \right )$ $\small \cdots \cdots \cdots$ $\small P_{n}^{\left ( n \right )}\left ( x_{0} \right ) = f^{n}\left ( x_{0} \right )$ the approximation is getting better and better.

To sum up, the polynomial you are looking for should meet the following conditions:

Explain how the above conversion is done, taking the second-order derivative in the third line above as an example:

Transformation of the first arrow: $\small P_{n}\left ( x \right )$ After finding the second derivative function $\small x_{0}$ , bring it in to get $\small {P_{n}}''\left ( x_{0} \right ) = 2!a_{2}$

Transformation of the second arrow: so $\small {f}''\left ( x_{0} \right ) = 2!a_{2}$ , so $\small a_{2} = \frac{1}{2!}{f}''\left ( x_{0} \right )$

The coefficients in the polynomial function $\small a$ can all $\small f\left ( x \right )$ be expressed by, then we get:

The error is $\small R_{n} \left ( x \right ) = f\left (x \right ) - P_{n}\left ( x \right )$ . Because a polynomial function is used to infinitely approximate a given function, there must be a small amount of error between the two.

****************************************************************************************************************************************

4. Definition of Taylor’s formula

So we get the definition of Taylor's formula:

If a function has derivatives up to order in some open interval containing $\small f\left ( x \right )$ , then for $\small x_{0}$ $\small \left ( a,b \right )$ $\small \left ( n+1 \right )$ $\small \forall x \in \left ( a,b \right )$

The remainder (i.e. error) $\small R_{n}\left ( x \right ) = \frac{f^{\left ( n+1 \right )}(\xi )}{\left ( n+1 \right )!}(x-x_{0})^{n+1}$ is $\xi$ between $\small x_{0}$ x and x. There are several ways to express the remainder of Taylor's formula. The previous table is called the Lagrangian remainder of the nth-order Taylor expansion. The Lagrangian remainder is the n-order Taylor formula expanded to one more order, and n becomes n+1. Note that the remainder here is the error, because if you use a polynomial function to expand at a certain point and approximate a given function, there will definitely be a small amount of error in the end, which we call the remainder.

****************************************************************************************************************************************

5. Extension – Maclaurin’s formula

It is a special case of Taylor's formula: the Taylor formula $\small x_{0} = 0$ at that time . So $\small x_{0} = 0$ putting it into the formula, we get:

Maclaurin's formulas with Peano remainders for several common elementary functions:

$\small \left ( x-x_{0} \right )^{n}$ The higher order infinitesimal of the Peano remainder is :