The method has the following data set into sklearn Method:
KFold,GroupKFold,StratifiedKFold,LeaveOneGroupOut,LeavePGroupsOut,LeaveOneOut,LeavePOut,ShuffleSplit,GroupShuffleSplit,StratifiedShuffleSplit,PredefinedSplit,TimeSeriesSplit,
① data set into --K fold cross-validation method: KFold, GroupKFold, StratifiedKFold,
- The entire training set S into k disjoint subsets, S is assumed that the number of training examples is m, then they have each a m / k training examples, the respective subset of S { . 1 , S 2 , ..., S K }
- Good points from each subset inside out as a test set, the k-1 other training set
- Learning model trained on the training set of the k-1
- The model is placed in the test set, the average value of the classification as the classification model or assumed that the true function of the rate
This method takes full advantage of so samples, but more complicated calculation, need to be trained k times, k test times
KFold:
import numpy as np #KFold from sklearn.model_selection import KFold X=np.array([[1,2],[3,4],[5,6],[7,8],[9,10],[11,12]]) y=np.array([1,2,3,4,5,6]) kf=KFold(n_splits=2) #分成几个组 kf.get_n_splits(X) print(kf)
for train_index,test_index in kf.split(X):
print("Train Index:",train_index,",Test Index:",test_index)
X_train,X_test=X[train_index],X[test_index]
y_train,y_test=y[train_index],y[test_index]
#print(X_train,X_test,y_train,y_test)
#KFold(n_splits=2, random_state=None, shuffle=False) #Train Index: [3 4 5] ,Test Index: [0 1 2] #Train Index: [0 1 2] ,Test Index: [3 4 5]
GroupKFold:
import numpy as np from sklearn.model_selection import GroupKFold X=np.array([[1,2],[3,4],[5,6],[7,8],[9,10],[11,12]]) y=np.array([1,2,3,4,5,6]) groups=np.array([1,2,3,4,5,6]) group_kfold=GroupKFold(n_splits=2) group_kfold.get_n_splits(X,y,groups) print(group_kfold) for train_index,test_index in group_kfold.split(X,y,groups): print("Train Index:",train_index,",Test Index:",test_index) X_train,X_test=X[train_index],X[test_index] y_train,y_test=y[train_index],y[test_index] #print(X_train,X_test,y_train,y_test) #GroupKFold(n_splits=2) #Train Index: [0 2 4] ,Test Index: [1 3 5] #Train Index: [1 3 5] ,Test Index: [0 2 4]
StratifiedKFold: to ensure that the proportion of each type of training set is the same
import numpy as np from sklearn.model_selection import StratifiedKFold X=np.array([[1,2],[3,4],[5,6],[7,8],[9,10],[11,12]]) y=np.array([1,1,1,2,2,2]) skf=StratifiedKFold(n_splits=3) skf.get_n_splits(X,y) print(skf) for train_index,test_index in skf.split(X,y): print("Train Index:",train_index,",Test Index:",test_index) X_train,X_test=X[train_index],X[test_index] y_train,y_test=y[train_index],y[test_index] #print(X_train,X_test,y_train,y_test)
#StratifiedKFold(n_splits=3, random_state=None, shuffle=False)
#Train Index: [1 2 4 5] ,Test Index: [0 3]
#Train Index: [0 2 3 5] ,Test Index: [1 4]
#Train Index: [0 1 3 4] ,Test Index: [2 5]
② data set into a method - a method leave: LeaveOneGroupOut, LeavePGroupsOut, LeaveOneOut, LeavePOut,
- Leave-one authentication (the Leave-One-OUT, LOO) : Suppose there are N samples, each sample as a test sample, the other N-1 samples as training samples thus obtained N classifiers, N test results, the mean value of the N results to measure the performance of the model
- If LOO compared with K-fold CV, LOO based on N samples of the N models instead of k, and further, each of which is trained on the N-1 of N samples of the model, rather than (- K- 1) * n / k. In both methods, assuming k is not large and k << N, LOO more than k-fold CV Processed
- P left authentication method (the Leave-P-OUT) : there are N samples, each sample as a test specimen P, NP other samples as training samples thus obtained a train-test pairs, and unlike LeaveOneOut KFold, when P > 1, the test set will overlap, when P = 1, when it becomes the leave-one
leaveOneOut: a test set to leave
import numpy as np from sklearn.model_selection import LeaveOneOut X=np.array([[1,2],[3,4],[5,6],[7,8],[9,10],[11,12]]) y=np.array([1,2,3,4,5,6]) loo=LeaveOneOut() loo.get_n_splits(X) print(loo) for train_index,test_index in loo.split(X,y): print("Train Index:",train_index,",Test Index:",test_index) X_train,X_test=X[train_index],X[test_index] y_train,y_test=y[train_index],y[test_index] #print(X_train,X_test,y_train,y_test)
#LeaveOneOut()
#Train Index: [1 2 3 4 5] ,Test Index: [0]
#Train Index: [0 2 3 4 5] ,Test Index: [1]
#Train Index: [0 1 3 4 5] ,Test Index: [2]
#Train Index: [0 1 2 4 5] ,Test Index: [3]
#Train Index: [0 1 2 3 5] ,Test Index: [4]
#Train Index: [0 1 2 3 4] ,Test Index: [5
LeavePOut: leave the P test set
import numpy as np from sklearn.model_selection import LeavePOut X=np.array([[1,2],[3,4],[5,6],[7,8],[9,10],[11,12]]) y=np.array([1,2,3,4,5,6]) lpo=LeavePOut(p=3) lpo.get_n_splits(X) print(lpo) for train_index,test_index in lpo.split(X,y): print("Train Index:",train_index,",Test Index:",test_index) X_train,X_test=X[train_index],X[test_index] y_train,y_test=y[train_index],y[test_index] #print(X_train,X_test,y_train,y_test) #LeavePOut(p=3) #Train Index: [3 4 5] ,Test Index: [0 1 2] #Train Index: [2 4 5] ,Test Index: [0 1 3] #Train Index: [2 3 5] ,Test Index: [0 1 4] #Train Index: [2 3 4] ,Test Index: [0 1 5] #Train Index: [1 4 5] ,Test Index: [0 2 3] #Train Index: [1 3 5] ,Test Index: [0 2 4] #Train Index: [1 3 4] ,Test Index: [0 2 5] #Train Index: [1 2 5] ,Test Index: [0 3 4] #Train Index: [1 2 4] ,Test Index: [0 3 5] #Train Index: [1 2 3] ,Test Index: [0 4 5] #Train Index: [0 4 5] ,Test Index: [1 2 3] #Train Index: [0 3 5] ,Test Index: [1 2 4] #Train Index: [0 3 4] ,Test Index: [1 2 5] #Train Index: [0 2 5] ,Test Index: [1 3 4] #Train Index: [0 2 4] ,Test Index: [1 3 5] #Train Index: [0 2 3] ,Test Index: [1 4 5] #Train Index: [0 1 5] ,Test Index: [2 3 4] #Train Index: [0 1 4] ,Test Index: [2 3 5] #Train Index: [0 1 3] ,Test Index: [2 4 5] #Train Index: [0 1 2] ,Test Index: [3 4 5]
③ Method data set into - random partitioning method: ShuffleSplit, GroupShuffleSplit, StratifiedShuffleSplit
- ShuffleSplit iterator produce a specified number of independent of train / test data set into first random sample of all disrupted, and then divided into train / test to be used to control the random number seed random_state digital sequence generator so that the information can be calculated results reappear
- ShuffleSplit is a better alternative KFlod cross-validation, the better he allowed the sample iterations proportional control and train / test of
- StratifiedShuffleSplit and ShuffleSplit of a variant return hierarchical division, that is, when creating division to ensure that the proportion of the original sample proportion in each class and division consistent with the overall data set
#ShuffleSplit data sets out of order, then the training set and test set division ratio of the amount of training set and test set randomly chosen, the ratio of training set and test set and may be less than 1
import numpy as np from sklearn.model_selection import ShuffleSplit X=np.array([[1,2],[3,4],[5,6],[7,8],[9,10],[11,12]]) y=np.array([1,2,3,4,5,6]) rs=ShuffleSplit(n_splits=3,test_size=.25,random_state=0) rs.get_n_splits(X) print(rs) for train_index,test_index in rs.split(X,y): print("Train Index:",train_index,",Test Index:",test_index) X_train,X_test=X[train_index],X[test_index] y_train,y_test=y[train_index],y[test_index] #print(X_train,X_test,y_train,y_test) print("==============================") rs=ShuffleSplit(n_splits=3,train_size=.5,test_size=.25,random_state=0) rs.get_n_splits(X) print(rs) for train_index,test_index in rs.split(X,y): print("Train Index:",train_index,",Test Index:",test_index)
#ShuffleSplit(n_splits=3, random_state=0, test_size=0.25, train_size=None)
#Train Index: [1 3 0 4] ,Test Index: [5 2]
#Train Index: [4 0 2 5] ,Test Index: [1 3]
#Train Index: [1 2 4 0] ,Test Index: [3 5]
#==============================
#ShuffleSplit(n_splits=3, random_state=0, test_size=0.25, train_size=0.5)
#Train Index: [1 3 0] ,Test Index: [5 2]
#Train Index: [4 0 2] ,Test Index: [1 3]
#Train Index: [1 2 4] ,Test Index: [3 5]
#StratifiedShuffleSplitShuffleSplit data sets out of order, and then divide the test set and the training set, the ratio of the amount of training set and a test set of randomly selected proportion of the training and test sets, and can be less than 1, but also to ensure that all types of training set the proportion is the same
import numpy as np from sklearn.model_selection import StratifiedShuffleSplit X=np.array([[1,2],[3,4],[5,6],[7,8],[9,10],[11,12]]) y=np.array([1,2,1,2,1,2]) sss=StratifiedShuffleSplit(n_splits=3,test_size=.5,random_state=0) sss.get_n_splits(X,y) print(sss) for train_index,test_index in sss.split(X,y): print("Train Index:",train_index,",Test Index:",test_index) X_train,X_test=X[train_index],X[test_index] y_train,y_test=y[train_index],y[test_index] #print(X_train,X_test,y_train,y_test) #StratifiedShuffleSplit(n_splits=3, random_state=0, test_size=0.5,train_size=None) #Train Index: [5 4 1] ,Test Index: [3 2 0] #Train Index: [5 2 3] ,Test Index: [0 4 1] #Train Index: [5 0 4] ,Test Index: [3 1 2]