Convex Functions
The definition of a convex function1 is as follows:
As shown in the figure below: Strictly convex function: the function curve is located below the straight line formed by the points and connections.
Convex function: The curve of the function does not exceed the straight line formed by the points and connections.
Theorem 1: If a function is second-order differentiable in a certain interval and the second-order derivative is non-negative, then this function is convex in this interval.
Where twice differentiable refers to the second-order derivability.
The proof of this theorem is as follows:
Corollary 1: -ln(x) is a strictly convex function on (0,∞).
The proof is as follows:
Among them, Definition 2 is the definition of concave function.
Jensen's inequality
Theorem 2: Jensen's inequality:
The corollary of the above theorem describes two points. If we look at n points, we get Jensen's inequality.
The proof is as follows, using induction:
Because -ln(x) is a convex function, we take the -ln(x) function as f(x) and get:
This inequality is used in the EM algorithm.
Inference 2: The arithmetic mean is greater than or equal to the geometric mean
The proof is as follows: