Series Article Directory

[Artificial Intelligence Study Notes] Mathematics in Artificial Intelligence - Overview
[Mathematics in Artificial Intelligence] Differential Calculus of One Variable Function
[Mathematics in Artificial Intelligence] Basic Linear Algebra
[Mathematics in Artificial Intelligence] Multivariate Function Differential

foreword

Compared with software development, the field of artificial intelligence requires a lot of mathematical knowledge. Mainly calculus, linear algebra, probability theory and optimization are covered.
This article mainly introduces the basics of linear algebra.
This article serves as my notes on learning artificial intelligence, mainly for myself to review the past and learn the new in the future, and sorting it out here is considered a second study. It's an honor to be of help to you. If I am wrong, I welcome corrections. If there is any infringement, please contact the author to delete.

1. Vectors and their operations

Vector is the most basic concept in linear algebra. It is actually a one-dimensional array consisting of N numbers.
X = (X1 X2 . . Xn)
The geometric meaning of a vector is a point in space, and the physical meaning is speed or force. Vectors,
insert image description here
the components of vectors are called dimensions, and the whole set of n-dimensional vectors constitutes an n-dimensional Euclidean space:

1.2 Row and column vectors

Row vectors arrange vectors by rows, and column vectors arrange vectors by columns.
insert image description here
In mathematics, we write more data as column vectors, and in programming languages, we store more data as row vectors.

1.3 Vector operations

Vector operations mainly include: addition, multiplication, subtraction, inner product, and transpose. Let's list them one by one:

1.3.1 Addition and subtraction of vectors

The components that are equal to them are added separately. Obviously, the lengths of the two vectors must be equal. We will not list the subtraction here, and it is easy to infer other cases from one instance.
insert image description here

1.3.2 Multiplication of vectors

It is a number and each component of this vector is multiplied
insert image description here

1.3.3 Transpose

Convert a column vector to a row vector and a row vector to a column vector
insert image description here

1.3.4 Algorithms

A+B+C=A+(B+C)
K*(X+Y)=KX+KY

1.3.5 Inner Product of Vectors

Two column vectors:
insert image description here
equal to the multiplication of corresponding positions and then addition
The essence of the inner product of two vectors is to become a scalar

1.4 Norm of vectors

The formula of the norm is the absolute value of each component of the vector to the power of P and then use the power function to calculate one part of P. Here P must be an integer
1, 2, 3... to positive infinity.
The norm of the vector is to change the vector into a scalar, the expression of the norm is represented by two vertical lines, and then write P in the lower right corner
insert image description here

The norm is the sum of absolute values, and the 1st order norm is written as L1
The norm is the sum of the square and the root sign. In fact, it represents the length of the vector. The modulus of the vector learned in high school. It is very useful to write the 2 norm as the L2

norm later. It will be used when talking about regular terms later.

1.5 Special vectors

1.5.1 Zero vector

is a vector with all zero components
(0 0 . . 0)

1.5.2 Unit vectors

It is a vector with L2 norm/modulus/length 1

2. Matrix and its operations

insert image description here
A matrix is a two-dimensional array. It is an m-by-n matrix. It has m rows and n columns. Each row and column has an element on it. Each element has a row label i and a column label j, a ij

2.1 Square matrix, symmetric matrix, identity matrix, diagonal

2.1.1 Square matrix

Square matrix: The following introduces several special matrices. If m is equal to n, it is called a square matrix
insert image description here

2.1.2 Symmetric matrices

Symmetric matrix: the definition is that a ij is equal to a ji , then it is a symmetrical matrix, and it must be a square matrix
insert image description here

2.1.3 Identity matrix

Unit matrix: the main diagonal is all 1, and the other positions are 0. This is called the unit matrix. The unit matrix is written as I, which must be a square matrix, which is equivalent to 1 in the number.
insert image description here

2.1.4 Diagonal matrix

Diagonal matrix: the main diagonal is non-zero, and other positions are 0
insert image description here

2.2 Operation of matrix

2.2.1 Matrix addition and subtraction

The addition of the matrix is the addition of the corresponding positions of the matrix, and the same is true for the subtraction, which is the subtraction of the corresponding positions.
insert image description here

2.2.2 Multiplication

insert image description here
There is also a very special operation on matrices

2.2.3 Transpose

The operation of transposition is the same as that of vector, that is, to change aij into aji, and to exchange rows and columns
insert image description here

2.2.4 Multiplication of matrices

The matrix multiplication is not the same as the general multiplication.
It takes each row of the first matrix and each column of the second matrix to do the inner product to obtain the result, which
insert image description here
satisfies the distributive law, associative law, and commutative law
A+ B+C=A+(B+C)
addition must be satisfied, focus on multiplication
First of all, multiplication satisfies associative law
(AB) C=A(BC)
satisfies distributive law, here is left distributive law, and right distributive law
( A+B)C=AC+BC
A(B+C)=AB+AC
is particularly emphasized that the matrix does not satisfy the commutative law, not necessarily equal, even if the size of AB is different from that of BA,
AB≠
BA There is a special transpose formula
insert image description here

2.3 Inverse Matrix

The matrix has AB, but there is no A/B. In other words, there is only the inverse
matrix. How is the inverse matrix defined?
Suppose there is a matrix A, note that it must be a square matrix, multiplied by matrix B is equal to I
AB=I
or
BA=I I
is the identity matrix, then we call B here the right inverse matrix of A, and the left inverse matrix
has a A very important conclusion is that if such a B exists, its left inverse and right inverse must be equal.
What is the use of inverting the -1 matrix collectively called A ? It can help us solve linear equations. For example, if AZ=B
is multiplied by the inverse of A at the same time, then Z=A's -1 is multiplied by B. The purpose of its invention is also to do such things.
From here we can also see The identity matrix is like 1 in our multiplication.
Let's take a look at the formula:
insert image description here

2.4 Determinant

In fact, the determinant is not used much in machine learning. A matrix must be a square matrix to calculate its determinant. The determinant is the calculation method of
turning the matrix into a scalar . Let’s look at the determinant below.
insert image description here

The nature of the formula, of course, this is the number multiplied by α in terms of a square matrix

, which is equivalent to multiplying the nth power of α by the determinant of A, because when we looked at the calculation method just now, it is equivalent to multiplying each column by α. Is it n order?

insert image description here

Summarize

The above is what I will talk about today. This article only briefly introduces the basic knowledge of linear algebra, which belongs to the relatively basic knowledge points in college mathematics. I will continue to explain the advanced knowledge of linear algebra later.

[Mathematics in Artificial Intelligence] Basics of Linear Algebra