When the amount of data is small training, development, test set can follow 7: dividing ratio of 2, but the validation set when a huge amount of data (also called development set dev) and test set increases the amount of data of the model: 3 or 6: 2 not much to enhance the role of the training set should be allowed to have more data, such as 98 can be considered when 1 million the amount of data: 1: 1 or 99: 1.
Develop and test sets should use the same distribution as possible, but when less data will also be its practical application as much as possible to develop and test sets, so as to reflect the actual effect of a more objective model. Such as online crawling lots of pictures, user pictures taken less, the test set should use all or most of the pictures taken by the user in the cat category. This idea is similar to transfer learning.
After the purpose of the test set is complete the system development, performance evaluation system to help