Optimization Theory - (1) Introduction 2 Preliminary Mathematics Knowledge

1. The norm of vectors and the positive definiteness of matrices

1. Basic concepts

2. Inner product and norm

• norm, is a function with the concept of "length". In linear algebra, functional analysis, and related mathematics, a function that gives all vectors in a vector space a nonzero positive length or magnitude. Use ||x||
• Regarding norms and inner products, we have two important inequalities
  • Triangle inequality:
    $$||x+y||<=||x||+||y||$$
  • Cauchy's inequality:
    $$|x^T|<=||x||||y||$$
• definition
  
• theorem
  

3.Positive definite matrix

• Definition: Let A be a real symmetric square matrix of order n. In
  addition to any n-dimensional non-zero vector x, A is called an indefinite matrix.
• nature
  

2. Gradient of multivariate function, Hesse matrix and Taylor formula

1. Definition of gradient

2. Directional derivative






3. Gradient properties

4. Hesse matrix


5. Gradient formulas of several commonly used vector-valued functions

6. Differentiability of vector-valued functions


7. Taylor expansion of n-ary functions at one point