Deep learning algorithms/models-general chapter

Deep learning seems simple, and there are many ready-made models and encapsulated APIs, coupled with Google's free GPU, there is no problem in running a deep learning. However, when I need to design a network by myself, I find that I don’t know how to start (Flatten or GlobalAveragePooling2D? How to combine several layers of convolution and several pooling?). When I need to use it, I find that many concepts are paradoxical and I didn’t understand clearly ( How do artificial neurons simulate human neurons? How does convolution simulate the human visual cortex?), and the parameters in the model can be described by a dazzling array (how to set the learning rate and batch size?), plus In many cases, it is not possible to intuitively explain the changes in the results caused by the changes in some parameters. All of them indicate that behind all this simplicity, a solid theoretical foundation is needed, and what is also needed is a heart to understand the essence of things.

The neural network is mainly composed of three parts, namely:

• Network structure — describe the level of neurons and the structure of connecting neurons.
• Activation function (activation function)-used to add nonlinear factors to solve problems that cannot be solved by linear models.
• The choice of parameter learning method (usually weight value W and bias term b)-such as BP algorithm, etc.

The following will mainly describe these aspects.

0. Part 1: Origin

• The origin and development of neural networks

1. Introduction to Deep Learning

1.1 Basic concepts

• Deep learning basics

1.2 Introduction to the model

• Convolution basics and common convolution models

2. Deep learning model

2.1 Basic model

theory:

• Two Keras Deep Learning Strategies (1): Ten Convolutional Neural Network (CNN) Frameworks-Summary and Visual Analysis

Practical records:

• Keras deep learning strategy two articles (2): the actual use of migration neural network

2.2 Advanced model

3. Activation function and parameter learning

4. Summarize by yourself (take it out separately later)

Source of structural inspiration: a senior blogger

From neural network (NN) to convolutional neural network (CNN)

• Convolutional Neural Network CNN Summary: From Neural Network to Convolutional Neural Network (CNN)

From LeNet to DenseNet

• CNN network architecture evolution: from LeNet to DenseNet

How to quickly build a network such as vgg and Inception modules according to your own needs

• How to Develop VGG, Inception and ResNet Modules from Scratch in Keras

There is also knowledge of image processing and so on. .

reference:

1. Structure: "Neural Networks and Deep Learning"-Qiu Xipeng
2. Deep Learning (deep learning) study notes finishing series
3. "Illustrated Deep Learning"
4. Convolutional Neural Networks (CNN) for deep learning