Review the basics of linear algebra
Machine learning requires some basic knowledge of linear algebra.
matrix: Matrix
\[ A= \begin{bmatrix} 1402 & 191\\ 1371 & 821\\ 949 & 1437\\ 147&1448\\ \end{bmatrix} \]
\[ B= \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ \end{bmatrix} \]
- A is a \ (4 \ times2 \) a matrix consisting of four rows and two columns, and enclosed by the two brackets. Referred to as \ (. 4 {^ R & lt \ times2} \) .
- B is a \ (2 \ times3 \) matrix by two rows and three columns, and is enclosed by the two brackets. Referred to as \ (R & lt ^ {2 \ times3} \) .
- \ (A_ {ij} \) is used to indicate an element of a matrix, where \ (I \) line represents the matrix. \ (j \) column represents the matrix
- \(A_{11}=1402\)
- \(A_{12}=191\)
- \(A_{132}=1437\)
- \(A_{41}=147\)
- \(A_{43}=undefined\)
vector: Vector
\[ y= \begin{bmatrix} 460\\ 232\\ 315\\ 178\\ \end{bmatrix} \]
\ (Y \) is a set of vectors, the vector can be viewed as a \ ({n \ times1} \ ) matrix. Where n = 4, it is referred to as \ (R & lt. 4 ^ {} \) .
\ (y_i \) is the first vector \ (i ^ {th} \ ) elements
- \ (Y_1 = 460 \)
- \ (Y_2 = 232 \)
- \ (Y_3 = 315 \)
Learning the language of senior friends must know, for example, c ++ STL standard library in the vector index is counted from zero. In real life people learn, most people are used to from the beginning. Therefore, learning machine learning, we generally use 1 as the start, and in the preparation of program implementation, then switch back to 0.
Attach a period of MATLAB program
% The ; denotes we are going back to a new row. A = [1, 2, 3; 4, 5, 6; 7, 8, 9; 10, 11, 12] % Initialize a vector v = [1;2;3] % Get the dimension of the matrix A where m = rows and n = columns [m,n] = size(A) % You could also store it this way dim_A = size(A) % Get the dimension of the vector v dim_v = size(v) % Now let's index into the 2nd row 3rd column of matrix A A_23 = A(2,3)A = 1 2 3 4 5 6 7 8 9 10 11 12 v = 1 2 3 m = 4 n = 3 dim_A = 4 3 dim_v = 3 1 A_23 = 6
Matrix addition
\[ \begin{bmatrix} 1 & 0 \\ 2 & 5 \\ 3 & 1 \\ \end{bmatrix} + \begin{bmatrix} 4 & 0.5 \\ 2 & 5 \\ 0 & 1 \\ \end{bmatrix} = \begin{bmatrix} 5 & 0.5 \\ 4 & 10 \\ 3 & 2 \\ \end{bmatrix} \]
There are examples of adding a matrix above.
First, the two matrices have the same dimensions, i.e., the same number of columns of the same number of rows.
Two matrix addition is the digital position will add up, then get a new matrix, and this matrix and the original two matrices same dimension.
Adding can not be in different dimensions, for example:
\ [\}. 1 & bmatrix the begin {0 \\ \\. 3. 5 & 2. 1 & \\ \ End bmatrix} + {\ bmatrix the begin {0.5}. 4 & \ \ 2 & 5 \\ \ end { bmatrix} = \ mathop {error} \]
Matrix Multiplication
\[ 3\times \begin{bmatrix} 1 & 0 \\ 2 & 5 \\ 3 & 1 \\ \end{bmatrix} = \begin{bmatrix} 3 & 0 \\ 6 & 15 \\ 9 & 3 \\ \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 2 & 5 \\ 3 & 1 \\ \end{bmatrix} \times3 \]
A matrix multiplication of the above example, note the real number matrix multiplication.
Is a direct result of the individual elements of the matrix is multiplied with a real number to obtain a new matrix of the same dimension must
For real matrix multiplication, it is to take or not take affect after results
Multiplication Division like:
\ [\ bmatrix the begin {}. 4. 3 & 0 \\ \\ &. 6 \ bmatrix End {} \ setminus. 4 = \ FRAC. 1 {{}}. 4 \ Times \ bmatrix the begin {0}. 3 & \ \ 6 & 15 \\ \ end { bmatrix} = \ begin {bmatrix} 1 & 0 \\ \ frac {3} {2} & \ frac {3} {4} \\ \ end {bmatrix} \ times3 \]
general practice
\[ \begin{eqnarray} & & 3 \times \begin{bmatrix} 1 \\ 4 \\ 2 \\ \end{bmatrix} + \begin{bmatrix} 0\\ 0 \\ 5 \\ \end{bmatrix} - \begin{bmatrix} 3 \\ 0 \\ 2 \\ \end{bmatrix} \setminus 3 \\ & = & \begin{bmatrix} 3 \\ 12 \\ 6 \\ \end{bmatrix} + \begin{bmatrix} 0 \\ 0 \\ 5 \\ \end{bmatrix} - \begin{bmatrix} 1 \\ 0 \\ \frac{2}{3} \\ \end{bmatrix}\\ & = & \begin{bmatrix} 2 \\ 12 \\ \frac{31}{3} \\ \end{bmatrix}\\ \end{eqnarray} \]
MATLAB code:
% Initialize matrix A and B A = [1, 2, 4; 5, 3, 2] B = [1, 3, 4; 1, 1, 1] % Initialize constant s s = 2 % See how element-wise addition works add_AB = A + B % See how element-wise subtraction works sub_AB = A - B % See how scalar multiplication works mult_As = A * s % Divide A by s div_As = A / s % What happens if we have a Matrix + scalar? add_As = A + sA = 1 2 4 5 3 2 B = 1 3 4 1 1 1 s = 2 add_AB = 2 5 8 6 4 3 sub_AB = 0 -1 0 4 2 1 mult_As = 2 4 8 10 6 4 div_As = 0.5000 1.0000 2.0000 2.5000 1.5000 1.0000 add_As = 3 4 6 7 5 4
Matrix vector multiplication
\[ \begin{bmatrix} 1 & 3 \\ 4 & 0 \\ 2 & 1 \\\end{bmatrix} \times\begin{bmatrix} 1 \\ 5 \\\end{bmatrix} =\begin{bmatrix} 16^{(1)} \\ 4^{(2)} \\ 7^{(3)} \\\end{bmatrix} \\\begin{eqnarray} 1 \times 1 + 3 \times 5 = 16 \tag{1}\\ 4 \times 1 + 0 \times 5 = 4 \tag{2}\\ 2 \times 1 + 1 \times 5 = 7 \tag{3}\\\end{eqnarray} \]
There is a particular example above, the display matrix vector multiplication equations and procedure
Multiplied by:
- Set matrix \ (A \) , vectors \ (B \) .
- \ (A_j B_i = \) (the number of columns equal to the number A of lines B)
Each row element of A and B is multiplied by one column, and a value obtained by adding.
The new number of rows of a matrix and obtained the same number of columns with the same vector.
You can refer to the following example:
\ [\} bmatrix the begin {\\ A & B & C D E & F \\\ \\ bmatrix End {} * \ bmatrix the begin {X} End {bmatrix \\\ \\ Y } = \ begin {bmatrix} a * x + b * y \\ c * x + d * y \\ e * x + f * y \\\ end {bmatrix} \]MATLAB code:
% Initialize matrix A A = [1, 2, 3; 4, 5, 6;7, 8, 9] % Initialize vector v v = [1; 1; 1] % Multiply A * v Av = A * vA = 1 2 3 4 5 6 7 8 9 v = 1 1 1 Av = 6 15 24
Matrix and matrix multiplication
We now calculate such a formula
\ [\ begin {bmatrix} 1 & 3 & 2 \\ 4 & 0 & 1 \\\ end {bmatrix} * \ begin {bmatrix} 1 & 3 \\ 0 & 1 \\ 5 & 2 \\\ end {bmatrix} \
] occupancy just learned vector matrix, the second matrix is split into two vectors
\ [\ begin {bmatrix} 1 & 3 & 2 \\ 4 & 0 & 1 \ \\ end {bmatrix} * \ begin {bmatrix} 1 \\ 0 \\ 5 \\\ end {bmatrix} = \ begin {bmatrix} 11 \\ 9 \\\ end {bmatrix} \]
\[ \begin {bmatrix} 1 & 3 & 2 \\ 4 & 0 & 1 \\\end {bmatrix} *\begin {bmatrix} 3 \\ 1 \\ 2 \\\end {bmatrix} =\begin {bmatrix} 10 \\ 14 \\\end {bmatrix} \]
In fact, we've done the calculation, differing only in the last step, the order of the original column will answer merge, you can get
\ [\ begin {bmatrix} 1 & 3 & 2 \\ 4 & 0 & 1 \\\ end {bmatrix} * \ begin {bmatrix} 1 & 3 \\ 0 & 1 \\ 5 & 2 \\\ end {bmatrix} = \ begin {bmatrix} 11 & 10 \\ 9 & 14 \\\ end {bmatrix} \]
Multiplied by:
- Matrix 1 is set \ (A \) , the matrix 2 \ (B \) .
- \ (A_j B_i = \) (the number of columns equal to the number A of lines B)
Each row element of A and B is multiplied by one column, and a value obtained by adding.
A same number of rows obtained new matrix B with the same number of columns. That \ (R ^ {m * n } \ times R ^ {n * o} = R ^ {m * o} \)
You can refer to the following example:
\ [\} bmatrix the begin {\\ A & B & C D E & F \\ \\ \ bmatrix End {} * \ bmatrix the begin {W} X & \\ \\ Y & Z \ end {bmatrix} = \ begin {bmatrix} a * w + b * y & a * x + b * z \\ c * w + d * y & c * x + d * z \\ e * w + f * y & e * x + f * z \\ \ end {bmatrix} \]MATLAB code:
% Initialize a 3 by 2 matrix A = [1, 2; 3, 4;5, 6] % Initialize a 2 by 1 matrix B = [1; 2] % We expect a resulting matrix of (3 by 2)*(2 by 1) = (3 by 1) mult_AB = A*B % Make sure you understand why we got that resultA = 1 2 3 4 5 6 B = 1 2 mult_AB = 5 11 17
Some properties of matrix multiplication
Can not be exchanged (in general)
Real number multiplication, the two numbers after the exchange is a result of the same sense:
\ [3 + 5 + 3 = 5 \]
which is normal to the. But it is also true matrix?We tried to use the above matrix multiplication look:
\ [\ bmatrix the begin {}. 1 & 0. 1 & 0 \\ \\ \ bmatrix End {} * \ bmatrix the begin {0} & 0 & 2 0 \\ \\ \ {End bmatrix} = \ begin {bmatrix} 2 & 0 \\ 0 & 0 \\ \ end {bmatrix} \]\[ \begin {bmatrix} 0 & 0 \\ 2 & 0 \\ \end {bmatrix} * \begin {bmatrix} 1 & 1 \\ 0 & 0 \\ \end {bmatrix} = \begin {bmatrix} 0 & 0 \\ 2 & 2 \\ \end {bmatrix} \]
See it, the result is not the same.
But this is generally the case, there is a case that can be exchanged.
Special circumstances exchangeable (Identity matrix)
There is a matrix we call matrix (Identity matrix), which is characterized by:
Matrix must be \ (n \ times n \) , denoted $ I \ space or \ space I_ {n \ times n} $
矩阵对角线一定是1，其他部分一定是0
\[ \mathop{ \begin {bmatrix} 1 & 0 \\ 0 & 1 \\ \end {bmatrix} }\limits_{2 \times 2} \space \space \space \space \space \space \space \space \space \space \mathop{ \begin {bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end {bmatrix} }\limits_{3 \times 3} \space \space \space \space \space \space \space \space \space \space \mathop{ \begin {bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \\ \end {bmatrix} }\limits_{4 \times 4} \space \space \space \space \space \space \space \space \space \space \mathop{ \begin {bmatrix} 1 & & & & & \\ & 1 & & & & \\ & & 1 & & & \\ & & & 1 & & \\ & & & & \ddots & \\ & & & & & 1 \\ \end {bmatrix} }\limits_{n \times n} \]
And multiplying the matrix, no matter how switching matrix, the result will not change.
MATLAB code:
% Initialize random matrices A and B A = [1,2;4,5] B = [1,1;0,2] % Initialize a 2 by 2 identity matrix I = eye(2) % The above notation is the same as I = [1,0;0,1] % What happens when we multiply I*A ? IA = I*A % How about A*I ? AI = A*I % Compute A*B AB = A*B % Is it equal to B*A? BA = B*A % Note that IA = AI but AB != BAA = 1 2 4 5 B = 1 1 0 2 I = Diagonal Matrix 1 0 0 1 IA = 1 2 4 5 AI = 1 2 4 5 AB = 1 5 4 14 BA = 5 7 8 10
The inverse matrix (inverse matrix)
The concept of reciprocal very familiar with it. And a number multiplied by another number 1 and so on and we think it's digital reciprocal.
\ [3 \ times (3 ^
{- 1}) = 1 \\ 5 \ times (5 ^ {- 1}) = 1 \\ \] for the matrix, we have the same concept. Since we believe that the same matrix and a real number in position 1, it is expressed:
\ [A (A ^ {- 1}) = (A ^ {- 1}) A = the I \]
we call \ (A ^ {--1} \) is the inverse matrix.
\ [\ Mathop {\ begin { bmatrix} 3 & 4 \\ 2 & 16 \\ \ end {bmatrix}} \ limits_A \ mathop {\ begin {bmatrix} 0.4 & -0.1 \\ -0.05 & 0.075 \\ \ end {bmatrix}} \ limits_ {A ^ {- 1}} = \ mathop {\ begin {bmatrix} 1 & 0 \\ 0 & 1 \\ \ end {bmatrix}} \ limits_ {AA ^ {- 1}} = I_ {2 \ times 2} \
] Some points to note:
- There is an inverse matrix of the matrix must be square
- \ (\ bmatrix the begin {} 0 & 0 & 0 \\ 0 \\ \ bmatrix {End} \) 0 matrix like this is no inverse matrix, because in any case can not let it become a unit matrix. You can not have the inverse square matrix approximation to zero look.
- No inverse matrix we call singular matrix or singular matrix
Matrix inversion
We now have a matrix:
\ [A = \ bmatrix the begin {}. 1 & 2. 3 & 0 \\ \\ &. 5. 9 & \ bmatrix End {} \]
and its inverted matrix is:
\ [A ^ T = \ begin {bmatrix} 1 & 3 \\ 2 & 5 \\ 0 & 9 \\ \ end {bmatrix} \]
This operation can be seen, each row vector of the same value into a column vector A, then the sequence spliced together.
\ (A \) After transposition, \ (A \) and \ (A ^ T \) corresponding relationship between each element is
\ [A ^ T_ {ij} = A_ {ji} \]MATLAB code
% Initialize matrix A A = [1,2,0;0,5,6;7,0,9] % Transpose A A_trans = A' % Take the inverse of A A_inv = inv(A) % What is A^(-1)*A? A_invA = inv(A)*AA = 1 2 0 0 5 6 7 0 9 A_trans = 1 0 7 2 5 0 0 6 9 A_inv = 0.348837 -0.139535 0.093023 0.325581 0.069767 -0.046512 -0.271318 0.108527 0.038760 A_invA = 1.00000 -0.00000 0.00000 0.00000 1.00000 -0.00000 -0.00000 0.00000 1.00000