Machine Learning - Linear Algebra - Inverse Map and Vector Space

Inverse maps and vector spaces

1. Inverse mapping

The essence of a matrix is ​​a map. For an $m\times matrix\; of\; n$ , multiplication$y=The\; function\; of\; Ax$ is to convert the vector fromnnxxin$n-\; dimensional\; original\; space$$x-$ coordinate position, mapped tommyy$in\; the\; m-$ dimensional target space$The\; y$ coordinate position, which is the process of forward mapping. Then, if the coordinate$y$ to deduce the coordinate$x$ , this process is called inverse mapping, because inverse mapping is also a mapping process, so the matrix representing the inverse mapping is called:$A^{-1}$。

It can be clearly seen from the above expression that the relationship between the original matrix and the inverse matrix is ​​bijective. This is also a necessary and sufficient condition for the existence of the inverse

1. Chunky matrix mapping is irreversible

The mapping compresses the original vector space so that it does not satisfy the injective condition: the original space has a many-to-one situation with respect to the target space

2. Tall and thin matrix mapping is irreversible

The target vector space cannot be completely covered by the original space vector, and the surjection condition is not satisfied: there is a many-to-one situation between the target space and the original space

3. Existence conditions of square matrix inverse mapping

When the column vectors in the square matrix are linearly related, combining all the conditions of the short and tall matrix, there must be no inverse mapping

When the column vectors in the square matrix are linearly related, there is an inverse mapping, why? There will be an explanation later.

2. Vector spaces and their subspaces

1. Vector spaces

1) The most common vector space: $R^n$

$R^{The\; n}$ space consists of allthe nnA column vector of$n\; components.$

2) General definition of vector space

Vector spaces are not limited to $R^n$ , of course we only discuss the vector space in the narrow sense (not discussing the generalized ones such as matrix, function space, etc. or extending to the complex field), then for a vector set$V$ , if any two vectors uuin V are taken$uwa$ vv$v$ , as long as$u+v$ still exist in$In\; V$ , take the scalar$c$ , as long as it satisfies$cu$ still in$V$ , then the set$V$ constitutes a vector space.

2. Subspace

Use the relationship between subsets and sets to compare the relationship between subspaces and vector spaces. If a vector space $U$ , his subset$V$ is also a vector space (satisfies the property requirements of vector addition and scalar multiplication), then$V$ isUUsubspace of$U.$

$m\times n$ matrix$A$ , four very important subspaces contained in the matrix: column space, null space, row space and left null space.

1) Column space

For matrix $As\; far\; as\; A$ is concerned, it contains$n$ mm_$m$ -dimensional column vector, then$The\; column\; space\; of\; A$ contains all the$n$ mm_$Linear\; combination\; of\; m-$ dimensional column vectors. Since each column is in$R^m$ space, and the sum of any two vectors in the column space and the quantitative product of any vector and any scalar can still be expressed as a linear combination of column vectors, which means that the column space C ( A ) C($C\left(A\right)$ is a vector space, and is$R^{A\; subspace\; of\; m-}$ space.

2) Null space

For an $m\times n$ matrix$As\; far\; as\; A$ is concerned, all satisfies$Ax=$vectorxx$of\; 0$$The\; set\; of\; x$ is called the matrix$The\; null\; space\; of\; A$ , denoted as$N\left(A\right)$ , if the matrixAAThe columns of$A$$x$ has only the unique solution of zero vector, ifAAThe columns of$A$$x$ has a non-zero solution.
The null space is also a vector space, for a$m\times For\; a\; matrix\; of\; n$ , the vectors in his null space are$n-$ dimensional, so the null space is$R^{}$subspace of${}^{n\; .}$

3) line space

For $m\times n$ matrix$A$ , its row space is the space formed by the vectors of each row of the matrix, then let's look at$The\; row\; vector\; of\; A$ is A n A^nafter transposition$A^{}$column vector of${}^{n\; .}$Therefore, it is not difficult for us to find that the matrix$The\; row\; space\; of\; A$ is ATA^Tafter transposition$A^{The\; column\; space\; of\; T}$ , denoted as:$C(A^T)$，$The\; row\; vector\; of\; A$ has$n$ components, so the row space is$R^{}$The subspace of${}^{n\; ,\; as\; for\; the\; proof\; of\; the\; establishment\; of\; the\; vector\; space,\; it\; will\; not\; be\; described\; again.}$

4) left null space

It will be easier to understand from the perspective of the transposed matrix. For $m\times n$ matrix$A$ , his left null space is the transpose matrix$A^{The\; null\; space\; of\; T}$ , which satisfies:$A^{T}x=0$ , we will denote it as$N(A^T)$.
The relevant properties of the left null space are not difficult to obtain, it is$R^{A\; subspace\; of\; m-}$ space.

Now we can answer why there is an inverse mapping when only the column vectors in the square matrix are linearly related.

If a matrix $A$ has an inverse mapping, which means that the mapped point can be uniquely restored, so obviously the null space$N\left(A\right)$ can not correspond to a one-dimensional straight line or a two-dimensional plane, but can only be a point, that is, 0 0in the original space$0$ vector, that is, if a matrix is ​​invertible, its$N\left(A\right)$ must be$0-$ dimensional.

If $The\; null\; space\; of\; A$ is not zero-dimensional, and there are more than one set of linearly combined zero vectors in the column space, that is, more than one vector in the original space is mapped to a zero vector, which does not conform to the one-to-one relationship.

3. Rank

The dimension of the spanning space of the matrix column vector or row vector, the number of linearly independent columns of the matrix.

1) Association between column space and null space

Matrix $A$ column space$The\; dimension\; of\; C\left(A\right)$ is the matrix$A$ rank$r$ .
Null space, geometrically speaking, the null space is the matrix$A$ maps to the vector space at the origin of the target space, which by definition must also be in the column space. In this$m\times$Under the action of the matrix mapping of$n$$x$ is compressed into$0$ (that is, the dimension of a point), then we can derive that$The\; x-$ dimensional region becomes a 0 0in the column space$0-$ dimensional point, so after the linear mapping of the matrix, the difference between the space dimensions before and after is also$x$ , because by$The\; original\; space\; where\; A$ is transformed is$n$ -dimensional, and the mapped column space is$r-$ dimensional, the difference between the dimensions of the two spaces$n-r$ is the dimension of space compression, according to what we said,$n-r$ is the null space$N\left(A\right)$ Dimensions.

2) Association between column space and row space

In a matrix, it is not difficult for me to find that the number of linearly independent row vectors is actually equal to the number of linearly independent column vectors.

Therefore, we can conclude that the row space $C(A^T)$dimension is also$r$。

3) Line space and left null space

From the above two associated statements, it can be known that $C(A^T$ ) is of dimension$r$ , while the matrix$A^{}$The original space before mapping corresponding to${}^T$$m$ , then the left null space$N(A^T)$is of dimension$m-r$。