After the machine learning model design, but also just the first step, a very important part of the post-adjustment model is Machine Learning
Said before the late adjustment model, and then sort out the basic steps of the machine model design
1. Analytical data: analysis, data processing
2. Design Model: nerve overlay module, activation function selection
3. loss function: to assess whether the model is good
4. optimization function: by minimizing the loss function, nerve adjustment parameter module, typically the GD (gradient descent), the SGD (stochastic gradient descent), Adam (Ada + Momentum + adaptive momentum)
The training set: using the training set, to adjust the parameters of the model of the design cycle by executing an optimization function
6. validation set: trained using a training set of parameters tested to see how the loss of output function
7. Test Set: Like the validation set, validation set is mainly due to the separation of the test set may not give, it is necessary to divide themselves validation set
When the poor performance of the model, we need to model how to improve it?
Improved test set
First, we have to ask ourselves, good or bad performance of the model in the test set (Training Set) in!
Before I look at the performance of the test set, but in fact if the model has been tested in focus are performing well, basically not likely to get a good result in the test set
So when the effect of the test set to get very good, we have the model which methods to improve it?
The number of neurons
In each layer of the network, to adjust the number of neurons suitable. In theory, a neuron has a number of layers can represent any relationship, it can be appropriate to increase the number of neurons. But it can not be set too high, extremes meet.
Activation function
sigmod、relu、tanh、softplus、softsign、linear
Loss function
MSE (mean square) cross entropy
Optimization function
GD、SGD、mini_batch、Adam
Which, mini_batch can be regarded as a special SGD, parameter update it after obtaining a batch, rather than the data is updated every SGD
Adam is an adaptive learning rate adjustment, from the optimum value, the larger the learning rate, the smaller the distance from the optimum value, the smaller learning rate
Improved test set
When the training set works well, the test set a bad effect, it means that the current model overfiting
We should make the test after a training set of parameters overfiting, consider following these methods to improve test set
Early Stopping
When the test set of circumstances arise loss function value drops as the number of training first, and then rise to a certain point, we focus on verification to find out the best training times the value of the loss, stopped early
Regularization
Add constraint term in the loss function parameters, the first parameter is usually paradigm and a second paradigm
Dropout
When training set training, lose some neurons, then the test set in normal training
Is equivalent to the game, increasing the difficulty of peacetime training, will play really relaxed timing