Chapter One
Deep learning and machine learning
Deep learning is a type of machine learning. Deep learning uses deep neural networks, which are composed of multiple neural network layers, each containing many neurons. The number of layers and parameters of deep neural networks is usually large, allowing them to handle more complex tasks and large-scale data.
- Traditional machine learning usually requires manual design of features, that is, extracting useful features from raw data and using them as input for learning and prediction.
- A key feature of deep learning is the ability to automatically learn feature representations. By stacking neural networks layer by layer, deep learning models can learn advanced feature representations from raw data without manually defining features.
Data determines the upper limit of the model, and preprocessing and feature extraction are the core.
Deep learning solves the problem of how to extract features.
1. Neural Network
Mapping f(x,W) from input-》to output. w is the weight parameter.
f(x,W)=Wx+b, training changes w to minimize the value of the loss function L.
Why can neural networks solve nonlinear problems?
A nonlinear activation function is used. For example tanh.
1. Activation function
The most common activation functions are
-
Sigmoid
is also called the squeeze function. It is generally used for binary classification, but when saturated, the gradient will disappear; optimization time is large. -
fishy
-
relu
max (0, x)
optimizes quickly and converges quickly. But it cannot be learned when x<0. Commonly used. -
leaky relu
max (0.1x, x)
is very commonly used. Solved the problem of relu. -
maxout
-
up
2. Feedforward neural network
