The kernel technique uses the kernel function to directly calculate the inner product of two vectors mapped to high dimensions, thereby avoiding the high-dimensional mapping step. This article uses the concept of matrix to introduce the necessary and sufficient conditions of the kernel function $K(x,y)$: symmetric (semi-) positive definite.
Symmetric positive definite looks like a condition for matrices. In fact, for the function $K(x,y):\R^n\times \R^m\rightarrow \R$, all real values of the vector $x\in \R^n$ are regarded as matrices in order The row number of the vector $y\in \R^m$ is regarded as the column number of the matrix in order, and the element value corresponding to the row and column number is the corresponding function value $K(x,y)$, then You can get a matrix $K$ of infinite width and height with shape $n\infty\times m\infty$.
sufficiency
As introduced below, if the function $K(x,y)$ (or matrix $K$) is symmetric and positive definite, it can become a kernel function (i.e. positive definite kernel), that is, there is $\phi(x):\R^n\ rightarrow \R^N$, such that $K(x,y)=<\phi(x),\phi(y)>$.
Since the matrix $K$ is symmetric and positive definite, orthogonal decomposition can be performed
$K=Q\Lambda Q^T$.
Among them, the eigenvector matrix $Q=[q_1,q_2,...,q_{n\infty}]$, each vector $q_i$ has $n\infty$ dimensions and can be regarded as a function $q_i(x) :\R^n\rightarrow \R$. The eigenvectors are unit orthogonal, so $q_i^Tq_i=1$, $q_i^Tq_j=0$ (or $\int q_i(x)q_j(x)dx=0$). Eigenvalue matrix $\Lambda=\text{diag}(\lambda_1,...,\lambda_{n\infty})$. Then $K$ can be expressed as
\begin{aligned}K&=\sum\limits_{i=1}^{n\infty}\lambda_iq_iq_i^T \\&=\sum\limits_{i=1}^{n\infty}\sqrt{\lambda_i}q_i\sqrt{\lambda_i}q_i^T \\&= [\sqrt{\lambda_1}q_1,...,\sqrt{\lambda_{ n\infty}}q_{ n\infty}]\cdot [\sqrt{\lambda_1}q_1^T,...,\sqrt{\lambda_{ n\infty}}q_{ n\infty}^T]^T\end{aligned}
Add indexes $x, y$ to the above matrix $K$ and vector $q_i$, and it will become a function, and we get
$K(x,y) = [\sqrt{\lambda_1}q_1(x),...,\sqrt{\lambda_{ n\infty}}q_{ n\infty}(x)]\cdot [\sqrt{\lambda_1}q_1(y),...,\sqrt{\lambda_{ n\infty}}q_{ n\infty}(y)]^T$
For $\phi(x)=[\sqrt{\lambda_1}q_1(x),...,\sqrt{\lambda_{n\infty}}q_{n\infty}(x)]$, it is $ K(x,y)=<\phi(x),\phi(y)>$.
It can be seen that the positive definite kernel can be regarded as a process of first mapping a certain dimension and then calculating the inner product. So how to judge whether a certain function $K(x,y):\R^n\times \R^n\rightarrow \R$ is positive definite?
The above proof is a kernel that is positive definite in the entire real number domain, called the Mercer kernel, while the commonly used positive definite kernel only requires that it be positive definite in a certain subset of the real number domain. Therefore, the conditions for the positive definite kernel are more relaxed than the Mercer kernel, and the positive definite kernel includes the Mercer kernel. Usually the data we process only belongs to a certain interval in the real number domain, and does not require such strict requirements as the Mercer core. The above proof assumes that the row and column numbers of the matrix are real numbers arranged in order. In fact, as long as the row and column numbers correspond one to one, it is also true out of order. Just multiply the same row and row exchange matrices on the left and right of matrix $K$. That is, the common proof requires making any Gram matrix positive definite.
necessity
It is proved below that if the function $K(x,y)$ is a kernel function, then its symmetry is positive definite.
According to the condition, $K(x,y) = <\phi(x),\phi(y)>$, where $\phi(x):\R^n\rightarrow \R^N$. then it can be expressed as
$K(x,y) = [\phi_1(x),...,\phi_N(x)]\cdot [\phi_1(y),...,\phi_N(y)]^T$
Converted into matrices and vectors:
$K=\sum\limits_{i=1}^N\phi_i\phi_i^T$
Among them $K\in\R^{n\infty\times n\infty},\phi\in \R^{n\infty}$. For any $x\in \R^{n\infty}$, we have
$xKx^T=\sum\limits_{i=1}^Nx\phi_i\phi_i^Tx^T=\sum\limits_{i=1}^Nx\phi_i(\phi_ix)^T\geq 0$
So the symmetry is positive definite.