Keras Deep Learning - Neural Network Basics

Get into the habit of writing together! This is the third day of my participation in the "Nuggets Daily New Plan · April Update Challenge", click to view the details of the event .

A neural network is a powerful learning algorithm inspired by the way the brain works. Similar to the way neurons are connected to each other in the brain, neural networks take inputs, pass them through the network through some function, and some neurons connected behind them are activated, producing outputs.

Architecture of a Simple Neural Network

Artificial neural networks are inspired by the way the human brain works. Essentially, it is an improvement over linear regression and logistic regression, where the neural network introduces a variety of nonlinear functions when computing the output. Furthermore, neural networks have great flexibility in modifying network architectures to solve problems across multiple domains with both structured and unstructured data.

The more complex the function, the better the ability of the network to fit the input data, and therefore the higher the accuracy of the prediction.

The typical structure of a feedforward neural network is as follows:

A layer in a neural network is a collection of one or more nodes (computational units), each node in a layer is connected to each node in the next layer. The input layer consists of the input variables needed to predict the output value.

The number of nodes in the output layer depends on whether we want to predict continuous or categorical variables. If the output is a continuous variable, the output layer is a node.

If the output is a classification of predicted classes for n classes, there will be n nodes in the output layer. Hidden layers are used to convert the values ​​of the input layer to values ​​in a high-dimensional space so that we can learn more about the data from the input. The nodes in the hidden layer work as follows:

In the above figure, $x_1, x_2, ..., x_n$is the independent variable, $x_0$is the bias term, similar to a linear equation $y=kx+b$in$b$ $b$， $w_1, w_2, ..., w_n$is the weight assigned to each input variable. if $a$ is one of the nodes in the hidden layer, then the calculation method is as follows:

$$a = f (\ sum _ {w_i} ^N w_ix_i)$$

  $f$ -function is an activation function that introduces non-linearity on the sum of the inputs and their corresponding weight values. Stronger nonlinear capabilities can be achieved by using multiple hidden layers.

In summary, a neural network is a collection of node weights in interconnected layers. The set is divided into three main parts: input layer, hidden layer and output layer. There can be n hidden layers in a neural network, and the term "deep learning" usually means a neural network with multiple hidden layers. Hidden layers, also known as intermediate layers (layers that are not input or output), are necessary when a neural network needs to learn a task with complex or non-obvious context, such as image recognition.

train the neural network

Training a neural network is actually about adjusting the weights in the neural network by repeating two key steps: forward propagation and back propagation. 

In forward propagation, we apply a set of weights to the input data, pass it to the hidden layer, use nonlinear activation on the computed output of the hidden layer, after passing through several hidden layers, compare the output of the last hidden layer with The other set of weights is multiplied together to get the result of the output layer. For the first forward pass, the values ​​of the weights will be randomly initialized.

In backpropagation, try to reduce the error by measuring the error of the output and then adjust the weights accordingly. The neural network repeats forward propagation and back propagation to predict the output until it obtains weights that make the error smaller.

Applications of Neural Networks

Recently, neural networks have been widely adopted in various applications. Neural networks can be constructed in a number of ways. Here are some common build methods:

The purple box at the bottom represents the input, followed by the hidden layer (yellow box in the middle), and the pink box at the top is the output layer. A one-to-one architecture is a typical neural network with hidden layers between the input and output layers. Examples of different architectures are as follows:

Architecture	Example
one-to-many	The input is an image and the output is the predicted class probability of the image
many-to-one	The input is a movie review and the output review is a good or bad review
many-to-many	Translate sentences in one language to sentences in another language using neural networks

 

An architecture that is often used in modern neural networks is called Convolutional Neural Networks (CNN), which can be used to understand the content in the image and detect where the content is located. The architecture is shown below (in the following Details will be introduced in the study):

Neural networks have a wide range of applications in recommendation systems, image analysis, text analysis and audio analysis. Neural networks can flexibly use a variety of architectures to solve problems. It is expected that the use of neural networks will become wider and wider. .