Within the vector product
This is basically a concept among high school mathematics textbooks, two vectors inner product is very simple, we look directly at the formula recap:
\[X \cdot Y = \sum_{i=1}^n x_i*y_i\]
Where X and Y are n-dimensional vectors, two vectors can be provided in the inner product of two vectors is the same dimension. Can be seen from the above formula, the inner product of two vectors is equal to the product of the two vectors corresponding to the respective components and dimensions.
Order and ordinary multiplication and matrix multiplication do distinction, we usually written as the product of two vectors: \ ([X, Y] Ty = X ^ \) .
There is a very important property for a vector, we can calculate its length with Euclidean formula. Further, we can use the angle between the vector and the vector length of the vector represents an inner product, as follows:
\ [[X, y] = | x | \ cdot | and | \ cos \ theta \]
Wherein \ (\ Theta \) is the angle between the vectors x and y, and three-dimensional vector in the space, it is very intuitive. For high-dimensional vector, it is difficult to imagine its physical meaning. But no, we can still think there is a vector between generalized Hyperspace an angle within. In the field of machine learning, we usually use this angle to reflect the similarity between vectors . The more similar the two vectors, then the angle between them should be as low, corresponding to the cosine cos should be larger. Therefore, we can use the cosine between two vectors to reflect the degree of similarity between them. Compute the cosine of the origins.
Orthogonal vector
Can be seen from the above equation, the vector inner product of two vectors is equal to the length multiplied by the angle between the vectors. For nonzero vector, the length thereof should be greater than zero. Therefore, the size of the inner product of two vectors, is completely dependent on the angle between the vectors \ (\ Theta \) .
If \ (\ Theta \) is less than 90 °, then the \ (\ COS \ Theta> 0 \) , then the inner product is positive. If \ (\ Theta \) is greater than 90 °, then the cosine is negative. So we can be positive or negative value determined by the cosine of the angle is an acute angle or an obtuse angle. Now comes angle, naturally inseparable from a special case - vertical .
If a two-dimensional plane in which the two vectors is the angle of 90 °, it is clear that two vertical vectors. The same is true in high-dimensional space which, but we generally do not say vertical, but will change a word - orthogonal . Inner product of two non-zero vector is 0, indicating two orthogonal vectors.
Orthogonal vectors
After figuring out the orthogonal vectors, orthogonal vectors also clear. Refers to a group of orthogonal vector set are orthogonal to each group and the non-zero vector .
If the n-dimensional vectors: \ (A_1, A_2, \ cdots, A_R \) are orthogonal to each, then they must be linearly independent. That is a set of coefficients is not zero does not exist \ (\ the lambda \) , such that:
\[\lambda_1a_1+\lambda_2a_2+\cdots + \lambda_ra_r=0\]
This is easy to prove, because the vectors within the volume is not zero, we just need to just ride on a vector to both sides of the equation, suppose we take a \ (A_1 \) . Because it are orthogonal to each other vector, the other items are all 0. If you want to set up the equation, it must be:
\ [\ Lambda_1a_1 Ta_1 ^ = 0 \]
Since \ (a_1 \) is not zero, then the \ (a_1 ^ Ta_1 \) necessarily is not 0, to make the equation holds, only \ (\ lambda_1 \) is 0.
Orthonormal basis
Our concept and the concept of orthogonal vectors based fusion, if the vectors \ (e_1, e_2, \ cdots , e_r \) is a basis vector space V. If among them orthogonal to each other , then they are called a set of orthonormal basis.
For a vector a, we can easily obtain the coordinate of each dimension in which the orthonormal basis:
\ [\ = Lambda_i e_i 'Ta = [a, e_i] \]
That vector a, in a dimension orthogonal coordinate specifications under certain groups, equal to it, and the dimension of the orthogonal basis vector inner product .
If we already know that the vector space V is a set of base \ (A_1, A_2, \ cdots, A_R \) , we ask how orthonormal basis V it?
Here to use an algorithm, called Schmidt algorithm . With this algorithm, we can find its orthogonal basis through a set of basis vectors space.
The algorithm is very simple, we can directly write its formula:
\[ \begin{aligned} b_1 &= a_1 \\ b_2 &= a_2 - \frac{[b_1, a_2]}{[b_1, b_1]}b_1\\ \cdots\\ b_r &= a_r - \frac{[b_1, a_r]}{[b_1, b_1]}b_1-\frac{[b_2, a_r]}{[b_2, b_2]}b_2 - \cdots - \frac{[b_{r-1}, a_r]}{[b_{r-1}, b_{r-1}]}b_{r-1} \end{aligned} \]
We just take two b-vector multiplication, we know, between two orthogonal vector b among the group. Therefore, we just look b vector set of units, it can be obtained the corresponding orthonormal basis.
which is:
\[ e_1=\frac{1}{||b_1||}b_1, \quad e_2=\frac{1}{||b_2||}b_2 \quad \cdots \quad e_r=\frac{1}{||b_r||}b_r \]
Although this algorithm is not difficult, but very important. In the field of machine learning some of dimensionality reduction algorithms , many of which are related to the Schmidt orthogonalization method.
Orthogonal matrix
Before we introduce the matrix in time, once said, we can put a matrix as a specific structure vector group. Similarly, we can put a set of orthonormal basis vectors as a matrix, then this matrix is called an orthogonal matrix .
It has the following properties:
\ [A = TA 'I \]
Where I is the identity matrix, it is necessary and sufficient condition that each column of the matrix A which is a unit column vector, and are orthogonal to each.
Finally, we look at the nature of the orthogonal matrix. Its main properties are three:
If A is an orthogonal matrix, then the \ (A ^ {-}. 1 A = ^ T \) , it is also an orthogonal matrix, and \ (| A | = \ PM. 1 \) .
If A and B are orthogonal matrices, and they are the same order, then AB is also orthogonal matrix.
If A is an orthogonal matrix, the vector y determinant unchanged after A conversion.
These three properties are very simple, we have by nature substantially orthogonal matrix can be deduced directly, or is very intuitive, and consistent with our intuition. In fact, how it is derived is not the point, for algorithm engineer, is more important to understand the meaning of these concepts, and it will play with the algorithm model which features linked to this is the most important thing.
Today content on orthogonal vectors and matrices on here, if that be harvested, please dig a point of concern or forward it, your support is my greatest motivation.