[Using Pytorch to implement an entry-level artificial neural network]

introduce

We all want to delve into deep learning and explore the various tasks it can perform, such as building a robot or translating Chinese to English. To delve deeper, we must start with the basics, the basic building blocks of neural networks that will help us how to process data, just like we do in our brains. This article will learn about neural networks from the ground up and how to build artificial neural networks using Pytorch.

Biological Neural Network (BNN):

Biological neural networks are the main inspiration for building artificial neural networks, simulating them to enable machines to perform complex tasks and think like humans. A neuron is the main unit of a neural network, consisting of a cell body called a soma, dendrites, and an axon. They receive signals from nearby neurons, and their axons carry the signals to other neurons. Contact between the terminals and the dendrites is achieved through synapses, where impulses are passed from one neuron to another like a domino effect.

Artificial Neural Networks (ANNs)

The artificial neural network is built on the idea of ​​imitating the way the human brain works, and it can be said to be a less complex mathematical model of the human brain. Like BNNs, neural networks have different characteristics, entering neurons with specific weights associated with their learning patterns, and firing outputs from neurons. These features are the input to neurons, like dendrites in BNN, the nodes are like nuclei, and the weights are represented by synapses with axons for output. An artificial neural network consists of an input layer, a hidden layer, and an output layer. The input layer accepts all inputs provided to it. A hidden layer sits between the input and output and is used to find patterns relevant to the question, this is the layer learned by updating its weights. The output layer can be a single neuron, such as two neurons to get the probability that the image is a cat or a dog in the case of predicting the next word in a sentence, or multiple neurons, for classification using softmax.

Artificial Neural Networks A neural network with multiple hidden layers is called a deep neural network. As more hidden layers are added to the network, it can go deeper, but too deep is not good because the network may suffer from a problem called degradation. There are many ways to build deep networks without running into this problem, such as using residual blocks. If you want to know more information, you can refer to my blog post "Using Pytorch to implement the ResNet network model: ResNet50, ResNet101 and ResNet152" .

How ANNs work

The input layer is connected to each hidden layer, and a weighted directed graph goes from hidden layer to output. These edges connect one neuron in a hidden layer to every other neuron in the next hidden layer. Each input is then multiplied by a corresponding weight, which typically represents the capabilities of a connection in a neural artificial network, and all weighted inputs are aggregated within the computational unit. If the weighted sum is zero, a bias term is used and summed to make the output non-zero. The bias takes the same input and the weight is equal to 1. Here the total input weight can be from 0 to any positive number. In neural networks, weights are updated during backpropagation.

In a feed-forward neural network, there is only one direction of value flow from input to output, and there is no weight update here, which is only used to provide output at one time. In the case of backpropagation, there is a loss function at the output, and the results we get in the output are compared in the loss function, and based on that, the weights are updated while backpropagating. In this way, the model learns to solve various specific tasks by updating the weights.

Implementing Artificial Neural Networks with PyTorch

To implement the artificial neural network, the python library of PyTorch is used in this example. PyTorch is widely used and almost all state-of-the-art models are currently implemented in it.

import torch
from torch import nn


class MyFirstNeuralNetwork(nn.Module):
    def __init__(self):
        super(MyFirstNeuralNetwork, self).__init__()
        self.flatten = nn.Flatten()
        self.net = nn.Sequential(
          nn.Linear(28*28, 512),
          nn.ReLU(),
          nn.Linear(512, 512),
          nn.ReLU(),
          nn.Linear(512, 10),
          nn.ReLU()
        )

    def forward(self, x):
        x = self.flatten(x)
        logits = self.net(x)
        return logits


if __name__ == '__main__':
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    model = MyFirstNeuralNetwork().to(device)
    print(model)

In the test code above, the output should be:

in conclusion

In this article, I mainly introduce the biological neural network and the artificial neural network that has been developed rapidly by imitating the biological neural network, and how to use the pytorch library to realize the artificial neural network. This is for your reference and learning. If there is something wrong , Please also correct me, thank you!