Deep Learning in the Browser: TensorFlow.js (3) More Basic Models

In the last blog , we introduced how to implement the simplest linear regression model. Today, let's take a look at how to use the same idea to implement more models.

logistic regression

Logistic regression is not only able to achieve binary classification. Let's take a look at an example of using logistic regression ( Multinomial logistic regression ) to achieve multi-classification.

This is the training data:

This is the result of classification. We can see that for some points, blue and orange, the classification results are better; for green and red points, the classification results are not very good.

The code is here:

function logistic_regression(train_data, train_label) {  
  const numIterations = 100;
  const learningRate = 0.1;
  const optimizer = tf.train.adam(learningRate);
  //Caculate how many category do we have
  const number_of_labels = Array.from(new Set(train_label)).length;
  const number_of_data = train_label.length;
  
  const w = tf.variable(tf.zeros([2,number_of_labels]));
  const b = tf.variable(tf.zeros([number_of_labels]));
  
  const train_x = tf.tensor2d(train_data);
  const train_y = tf.tensor1d(train_label);
  
  function predict(x) {
    return tf.softmax(tf.add(tf.matMul(x, w),b));
  }
  function loss(predictions, labels) {
    const y = tf.oneHot(labels,number_of_labels);
    const entropy = tf.mean(tf.sub(tf.scalar(1),tf.sum(tf.mul(y, tf.log(predictions)),1)));
    return entropy;
  }
  
  for (let iter = 0; iter < numIterations; iter++) {
    optimizer.minimize(() => {
      const loss_var = loss(predict(train_x), train_y);
      loss_var.print();
      return loss_var;
    })    
  }
}

The process of logistic regression is basically similar to the previous linear regression. There are several points to pay attention to:

training data
- train_x is each 2-dimensional matrix, and each point data is a binary array [x, y] consisting of x, y coordinates
- train_y is the classification of each point, starting from 0
The prediction model, for the softmax model, is simply an extension of binary logistic regression to more element vectors. If you are interested in learning more, you can read the following two articles:
Loss, the loss function uses cross entropy, you can refer to this article .
- Here, when calculating the loss, for the label, call the tf.oneHot() method to transform the label data into the following form
```
// Labels
Tensor
    [0, 0, 1, 1, 2, 2]

// OneHot
Tensor
    [[1, 0, 0],
     [1, 0, 0],
     [0, 1, 0],
     [0, 1, 0],
     [0, 0, 1],
     [0, 0, 1]]
```

K-nearest neighbors

K-nearest neighbor is a particularly simple algorithm, so simple that there is no training process. You can find a visual introduction to this algorithm in my other blog on machine learning .

This algorithm can also be implemented using TensorflowJS. The code below uses the L1 distance to implement the nearest neighbor algorithm (k=1)

function knn(train_data,train_label) {
  const train_x = tf.tensor2d(train_data);
  
  return function(x) {
    var result = [];
    x.map(function(point){
      const input_tensor = tf.tensor1d(point);
      const distance = tf.sum(tf.abs(tf.sub(input_tensor, train_x)),1);
      const index = tf.argMin(distance, 0);
      result.push(train_label[index.dataSync()[0]]);
    });
    return result;
  };
}

Reference: What are the various norms under machine learning?

Although this algorithm is simple, the amount of calculation is not small, and the prediction effect does not seem to be worse than logistic regression. We note the differences between similar data and logistic regression.

a math problem

Similarly, for general learning problems, TensorflowJS is naturally a no-brainer. Refer to the author's other blog: an example of using Tensorflow to solve geometric problems .

code show as below:

function train(train_data) {
  const numIterations = 200;
  const learningRate = 0.05;
  const optimizer = tf.train.sgd(learningRate);
  
  const training_data = tf.tensor2d(train_data);
  const center = tf.variable(tf.tensor1d([Math.random()* Math.floor(domain_max),Math.random()* Math.floor(domain_max)]));
  
  // Caculate the distance of this center point to the each point in the training data
  const distance = function() {
    return tf.pow(tf.sum(tf.pow(tf.sub(training_data, center),tf.scalar(2)),1),tf.scalar(1/2));
  }
  // Mean Square Error
  const loss = function(dis) {
    return tf.sum(tf.pow(tf.sub(dis,tf.mean(dis)),tf.scalar(2)));
  }
  for (let iter = 0; iter < numIterations; iter++) {
    var result = {};
    optimizer.minimize(() => {
      const loss_var = loss(distance());
      loss_var.print();
      result.loss = loss_var.dataSync();
      return loss_var;
    })
  }
  return center;
}

The operation effect is as follows: