As the successor of deeplearn.js, tensoflow.js supports running deep learning with JavaScript in the browser or nodejs. And can support GPU and existing Tensorflow models. There are several cool demos on the project's homepage. As a lover of machine learning and front-end data visualization, how can I not be tempted. Come and join me on how to use tensorflow.js on your deep learning journey.
Linear algebra is the mathematical foundation of deep learning. Tensor is the basic mathematical concept and operation unit of linear algebra. Let's take a look at the basic concepts and operations of tensors in tensorflow.js.
The concept of tensors
Scalar Scalar
A scalar is just a simple number, the concept is that there is only size and no direction.
const scalar = tf.scalar(5);
print_tensor([scalar]);Note: print_tensor() is a JS method I implemented that displays the content of the tensor in the form of a table in the browser, up to two dimensions. Input is an array of tensors. If you don't want to see the result in the browser, you can also call the tensor.print() method to view the content of the tensor in the console.
The above code creates a scalar with a value of 5 and a dimension (Rank) of 0.
Vector / Vector
A vector is a one-dimensional array, and the concept is that it has size and direction.
const vector = tf.tensor1d([0, 1, 2, 3, 4]);
print_tensor([vector]);
The above code creates a vector with the values [0,1,2,3,4] and dimension 1. Shape describes the capacity or size of each dimension, here is 5.
Tensor Tensor
A matrix (Matrix) has two dimensions, and in the higher dimension, it is a tensor. In fact, all scalars, vectors, and matrices can be represented by tensors, but the dimensions are different.
In Tensoflow, Rank represents the level of the dimension, which corresponds to the following
- scalar 0 tf.scalar
- Vector 1 tf.tensor1d
- matrix 2 tf.tensor2d
- Tensor2+ tf.tensor3d, tf.tensor4d
const matrix = tf.tensor2d([[1.0, 2.0, 3.0], [10.0, 20.0, 30.0]]);
print_tensor([matrix]);
The above code creates a matrix with the values [[1.0, 2.0, 3.0], [10.0, 20.0, 30.0]] and dimension 2. The shape is 2X3, two rows, three columns.
A picture to illustrate intuitively:
tensorflow.js provides methods for creating and generating , deforming , slicing and merging tensors for data preparation.
Operations on tensors
Tensor-scalar operations
The mathematical operations between tensors and scalars (addition, subtraction, multiplication, and division) are relatively simple, as long as each element of the tensor and the scalar are directly related to the mathematical operation.
const t = tf.tensor([[1.0, 2.0, 3.0], [10.0, 20.0, 30.0]]);
const s = tf.scalar(3);
print_tensor([t,s,t.add(s),t.mul(s)])
Mathematical operations on tensors and scalars do not change the shape of the tensors and satisfy the fundamental laws of arithmetic operations, such as the commutative law of addition, the associative law of multiplication, and so on.
Tensor - Tensor addition, subtraction and multiplication
Tensors with the same shape can be added and subtracted. Just add or subtract the corresponding elements.
const a = tf.tensor1d([1, 2, 3, 4]);
const b = tf.tensor1d([10, 20, 30, 40]);
print_tensor([a,b,a.add(b),a.sub(b),a.mul(b)])
tensor-tensor dot product
The product of tensors tensor.matMul() is widely used in deep learning. The following figure shows an example of a matrix dot product:
Simply put, the dot product of an m*n matrix and an n*k matrix yields an m*k matrix.
const a = tf.tensor2d([[1, 2], [3, 4]]);
const b = tf.tensor2d([[5, 6, 7], [7, 8, 9]]);
print_tensor([a, b, a.matMul(b)]); 
Note that the dot product of matrices does not satisfy the commutative law of multiplication, AB != BA
const a = tf.tensor2d([[1, 2], [3, 4]]);
const b = tf.tensor2d([[5, 6], [7, 8]]);
print_tensor([a, b, a.matMul(b), b.matMul(a)]); 
Matrix Transpose (Transpose)
The transpose of the matrix is to make the mirror inversion of all the elements of the matrix around a ray of 45 degrees to the lower right starting from the elements of the first row and the first column, that is, the transposition of the matrix is obtained.
const a = tf.tensor2d([[1, 2], [3, 4],[5,6]]);
print_tensor([a, a.transpose()]); 
Identity Matrix
Just like 1 in mathematics, any number multiplied by 1 is the number itself. The property of the identity matrix is that the result of the dot product of any matrix and the identity matrix is the original matrix itself. The identity matrix is an n*n matrix where all elements on the diagonal are 1, and the others are 0.
Use tf.oneHot() to generate an identity matrix.
const identity = tf.oneHot(tf.tensor1d([0, 1, 2]), 3);
print_tensor([identity]);
We will study the use of tf.initializers.identity to initialize the identity matrix later.
Inverse of a matrix
If one matrix is multiplied by another to obtain the identity matrix, then this matrix is the inverse of that matrix.
tensorflow provides the inverse operation of the matrix, tf.matrix_inverse
However, there is no corresponding operation in tensorflow.js. (Considering that the current version is 0.6.1, we put up with it. In addition, it is said that deep learning does not use any matrix inverse operations.)
Note that not all matrices are invertible, and some matrices cannot find an inverse.
To learn more about the inverse of a matrix, please refer to this article
The division of two matrices can be understood as the dot product of one matrix and the inverse of the other.
Reduction
In deep learning, it is often necessary to perform statistics on each dimension of a tensor. For example, mean, sum, etc. tensorflow.js provides corresponding operations.
const x = tf.tensor2d([[1, 2, 3],[4,5,6]]);
print_tensor([x, x.mean(),x.max(),x.min(), x.sum()]); 
The mean, maximum, minimum and sum of all elements of a matrix are given here, and corresponding operations can also be done for rows and columns.
The following code does the corresponding statistics for each column.
const x = tf.tensor2d([[1, 2, 3],[4,5,6]]);
print_tensor([x, x.mean(0),x.max(0),x.min(0), x.sum(0)]);
In addition to the operations mentioned above, tensorflow.js also provides:
- Basic mathematical operations such as trigonometric functions, logarithms, square roots, etc.
- logical operation
- ... ...
Summarize
This article describes the basic concepts and operations of tensors, and gives code examples in tensorflow.js. As the mathematical foundation of machine learning, especially deep learning, linear algebra is worth knowing and mastering.
All the code examples in this article can be run in my codepen .