In the previous article , the author introduced the basic concepts in TensorFlow.js, as well as the mathematical foundation of machine learning and the basic knowledge of linear algebra. In this article, let's take a look at how to use TensorFlow.js to build mathematical models and the basic process of learning.
The learning process is basically as follows:
- Prepare training data
- build a model
- Iterative learning using training data and models
- After the model is trained, use this model to predict new data (here we skip the validation part of the model)
Well, let's take the simplest linear regression as an example and see the process.
Prepare data
As you can see in the image above, I have generated 7 points in a 2D coordinate system that lie near some of my imaginary lines. I use these points as my training data.
The initialization code of the training data is as follows, where tx is the x coordinate of all point data, and ty is the coordinate of all point data.
const train_x = tf.tensor1d(tx);
const train_y = tf.tensor1d(ty);model selection
All models are wrong, and some models are better.
The so-called model, which is a function f, corresponds to a certain input data and calculates some output data. Models can be complex or simple. Simple models are not necessarily bad, and responsible models are not necessarily good either.
Let's take a linear model as an example, which mathematically assumes that Y = wX + b
In this model, there are two parameters to be determined, w and b.
Since the model is a function, its code is easy to understand:
const f = x => w.mul(x).add(b);Of course you can also write:
const f = function(x){
return w.mul(x).add(b);
}
}iterative learning
The learning process is called training. Training is usually an iterative process. In this process, the following things are usually required:
- A loss function, the loss function defines whether the model is good enough, usually the smaller the loss, the better.
- An optimizer that uses an algorithm to decide how to change the values of parameters so that the loss function is minimized.
- Iterative loop, through loop -> call optimizer, get new parameters, calculate loss, and finally when the loss is small enough, it can be considered that the training is over.
The training code is as follows:
Initialize the parameters, here random numbers are used as the initial values of the parameters. (Note that the initial parameters are not always chosen randomly.)
const w = tf.variable(tf.scalar(Math.random()));
const b = tf.variable(tf.scalar(Math.random())); initialize the learning parameters,
- numIterations is the number of iterations. Generally, the more the number of times, the better the fitting of the model, but the more computation is required.
- learningRate is the learning rate. The larger the value, the faster the learning speed, but it is also easier to miss the extreme point.
const numIterations = 200;
const learningRate = 1;Choose an optimizer, here I chose adam. TensorFlow.js provides a variety of optimizers , such as sgd, momentum, etc., you can choose according to your needs.
const optimizer = tf.train.adam(learningRate);For the loss function, we use the mean square error
const loss = (pred, label) => pred.sub(label).square().mean();or can be written as:
function loss(predictions, labels) {
const meanSquareError = predictions.sub(labels).square().mean();
return meanSquareError;
}Then there is the training process:
for (let iter = 0; iter < numIterations; iter++) {
optimizer.minimize(() => {
const loss_var = loss(f(train_x), train_y);
loss_var.print();
return loss_var;
})
}During the training process, we call the print() method of the tensor to print out the value of the loss to see if the training process converges. When the selected model, parameters, and optimizer are not suitable, it is possible that the training process does not converge.
As a result of training, we wait for the values of w and b. That is, the slope and intercept of the line are determined.
You can try my demo code
We can see how the learning process slowly converges to a straight line towards the final result.
Summarize
This article describes a process of learning the simplest linear regression model using tensoflow.js. I hope you can understand the basic idea of machine learning through this simple example.