1 training and validation sets
Training set (Train) : is used to train the neural network parameters that exact point, it is generally used gradient descent.
Validation set (Validation) : After each epoch is basically complete, to test the accuracy of the current model.
2 Why do you need a test set?
Validation set does not participate in training, there is no intersection with the training set, the accuracy rate is reliable, then why you need a test set (test) it?
In fact, for a model, its parameters can be divided into general parameters and super parameters . Without the introduction of intensive study, the general parameters that can be updated by gradient descent, which is updated training set of parameters. In addition, the concept of hyper-parameters, such as network layers, network nodes, the number of iterations, learning rate and so on, these parameters are not updated range of gradient descent. Although there are already some algorithms can be used to model the super-parametric search, but in most cases we still have to manually adjust according to their own validation set.
and so
That is to say, from the narrow sense, the validation set was not involved in the process of gradient descent, that is not trained; but from a broad point of view, the validation set was involved in an "artificial parameter adjustment" process , according to our the result of the validation set number of iterative adjustment, the learning rate adjustment, etc., such that the best results in the validation set . Therefore, we can also think, validation set is also involved in training.
So, we need a set of completely trained, and that is the test set, we neither test set gradient descent, do not have it to control the hyper-parameters, just after the final completion of the training model used to test the accuracy of the final.
3 endless process
however
The astute reader will analogy to, in fact, this is a never-ending process. If the test set accuracy is poor, then we would go to adjust various parameters of the model, this time it can be considered a test set is also involved in the training. Well, we may also need a "test test set", perhaps also need to "test test test set" ...
Forget it, or stop it in the test set.
4 division
If we do have a large data set of annotation, want to complete a supervised model tests, usually using a uniform random sampling, the data set into a training set, validation set, the test set, the three collections can not intersect, the common ratio is 8: 1: 1, the ratio is of course artificial. From this perspective, three sets are the same distribution.
If the game is done, the official label only provides a data set (training set) and a test set is not marked, so when we do the model, usually from the artificial division of the training set out a validation set. At this time we usually divide no longer a test set, there are two possible reasons: 1, the game is very basic to pull the party, the training set is inherently less; 2, we can not guarantee whether the test set to be submitted with training set completely identically distributed, so subdivided test set with a training set with distributed no much sense.
Reference: https: //www.jianshu.com/p/7e032a8aaad5