Softmax classification in TensorFlow and MNIST digit recognition

Table of contents

2. Understanding of Softmax classification

4. TensorFlow implementation of Softmax classification

1. Introduction

Deep learning has become one of the core technologies in the field of artificial intelligence, solving many problems including image recognition, natural language processing, and recommendation systems. In this blog, we will explore in depth how to use TensorFlow to implement Softmax classification and apply it to MNIST digit recognition, aiming to provide you with a simple and effective learning example.

2. Understanding of Softmax classification

Softmax classification is a commonly used method in multi-classification tasks. This function converts the raw output of each category into a probability form, so that the sum of the probabilities of all categories is 1, so that we can intuitively explain the output of the model. Its expression is:

$softmax(x)_i=\frac{e^{x_i}}{\sum_{j=1}^{n}e^{x_j}}$

where x is the original output of the model, and i represents the i-th class.

3. Introduction to MNIST dataset

MNIST (Modified National Institute of Standards and Technology database) is a widely used handwritten digit recognition dataset, which contains 60,000 training samples and 10,000 test samples. Each sample is a 28x28 grayscale image representing a digit from 0-9.

4. TensorFlow implementation of Softmax classification

First, we need to import the relevant libraries.

import tensorflow as tf
from tensorflow.keras.datasets import mnist

Next, load the MNIST dataset. TensorFlow provides a very convenient way to get the MNIST dataset.

(x_train, y_train), (x_test, y_test) = mnist.load_data()

Here, x_trainand y_trainare the images and labels of the training set, x_testand y_testare the images and labels of the test set.

Then, we need to preprocess the data. First convert the type of image data to floating point, and then divide by 255 for normalization. For labeled data, use tf.keras.utils.to_categoricalone-hot encoding.

x_train, x_test = x_train / 255.0, x_test / 255.0
y_train = tf.keras.utils.to_categorical(y_train)
y_test = tf.keras.utils.to_categorical(y_test)

Next, we need to define the model. Here we use a simple fully connected neural network where the activation function of the output layer is Softmax.

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dense(10, activation='softmax')
])

Next, we need to compile the model, specify the optimizer, loss function, and evaluation metrics.

model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

Then, we need to train the model.

model.fit(x_train, y_train, epochs=5)

Finally, we need to evaluate the performance of the model on the test set.

test_loss, test_acc = model.evaluate(x_test, y_test)
print('Test accuracy:', test_acc)

In the above code, we first use the tf.keras.datasets.mnist.load_data() function to load the MNIST dataset and scale the image pixel values between [0, 1]. Then, we use tf.keras.Sequential to build a simple Softmax classifier model, where the input layer is the Flatten layer, which flattens the input image into a one-dimensional vector; the middle layer is a fully connected layer, which maps the one-dimensional vector to a length of The output vector of 10; the output layer is a Softmax layer, which converts the output vector into a probability distribution. Next, we use the compile() function to compile the model, specifying the optimizer, loss function, and performance evaluation metrics. In this example, we use the adam optimizer, the sparse_categorical_crossentropy loss function, and the accuracy evaluation metric. Finally, we train the model using the fit() function and evaluate the model performance on the test set using the evaluate() function.

V. Summary

In this blog, we detailed the understanding and implementation of Softmax classification, and how to use TensorFlow for MNIST digit recognition. Hope it helps you.

Softmax classification and MNIST digit recognition are just the tip of the iceberg of deep learning. The application fields of deep learning are very broad, and we need to continue to explore in depth.

Note: The code in this article was written under TensorFlow 2.x version. If you are using TensorFlow 1.x version, some adjustments may be required.