[KnowingAI Zhizhi] Neural Network Related|Knowledge Card|Notes Collection

Neural Network by KnowingAI

UNDER

Neural Networks

• The emergence of neural networks is because people design a network structure that imitates the structure and working principle of neurons in the human brain by understanding the reasons for human thinking in order to realize "machine intelligence";
• The human brain can recognize a series of processes of apples-the retina receives light signals and converts them into corresponding electrical signals that are transmitted to the visual center through the cerebral cortex, which is determined by analyzing the color, contour, texture, shape and other characteristics of the object This is an apple; the principle of object recognition by neural network is an abstraction of the above process;
• A variety of neural networks-
  ①The input unit is a graph structure, then it is GNN
  ②The neural network is packaged, which is the Capsule Neural Network (CAPSNET) ③Convolution
  operation on the input data, so as to get CNN

Graph Neural Network (GNN)

• The reason for introducing GNN is that graph structure can express a wide range of meanings, so inputting graph as structure into neural network can be used to solve various problems;
• Research on different parts of the graph structure can also solve different problems-node attribute prediction, edge connection prediction, as well as the recognition and classification of the entire graph structure...
• GNN's research on speech and vision-text classification is because text can be regarded as a network composed of words; action recognition is because the human body can be regarded as a network of skeletons and fulcrums

In general, as long as there is a field of graph structure, you can consider applying GNN to solve the problem

CNN

• Set different convolution kernels to find different features in the original image

Capsule Neural Network

• When CNN processes images-
  ① it ignores the relative relationship between features and
  ② it is also difficult to understand operations such as rotation and scaling of things
  -so using CNN to train the network requires a lot of data to be able to recognize different positions , The same thing from the perspective
• Compared with CNN, the capsule network can better understand the composition, location and posture of things

RNN

• RNN-a network that attaches great importance to sequence information; in the language field, the order of different things will lead to huge differences in semantics, and because of the division of grammatical structure, different sequence positions often also imply relevant information, so RNN needs to be introduced to solve the problem .
• Working process: Each time a piece of data is input, in addition to calculating according to the network weight, the state of the neural network stored in the hidden state at the time of the last data input is also considered; as the data is input again and again, the stored information is constantly Update.
• RNN and natural language processing:
  ①Machine translation—find the expression of the same sequence of meanings in different languages;
  ②Poetry generation—based on a specific theme, output a logical sequence of words according to certain rules
  ps such as speech recognition, stock Quantitative transaction estimation, etc., these application scenarios that can link input data with time series can also be analyzed using RNN.

LSTM

• RNN uses hidden state to save the state information of the neural network at the last moment, but with the continuous input of data, the older the previous state information will occupy less and less weight, this is why RNN can only be used to retain short-term memory.
• The door is a small switch used to determine how to retain information, the value range is [0,1]; there are three doors attached to the updated hidden
  state—— ①Forgetting door: Decide how much original information is retained in the hidden state; (what old things are forgotten )
  ②Input gate: decide how much information in the current network state to keep; (remember what new things are)
  ③Output gate: decide how much information in the hidden state is output.
• The updated mechanism of LSTM: ①Understand
  the current network status through the input gate (short memory)
  ②Leave important information in the past through the forget gate (long memory)
• GRU (Gated Loop Unit)
  ①Update gate (forget gate + input gate): Decide to discard old information and add new information
  ②Reset gate: Decide on the network state at the last moment of writing (capture short-term memory)

The above content is derived from the original content of the B station up master @KnowingAI智智