This blog from the code in the "hands-on learning deep learning" pytorch version, the code is more, less of an explanation. However, many methods have mentioned in my previous blog, so this did not mention. Another reason is that this blog code, just take a look at certainly can understand (provided that the python syntax probably know) , this is my interpretation without a lot of important reasons.
K-fold cross-validation to achieve
DEF get_k_fold_data (K, i, X-, Y):
# Returns the training and validation data when the i-fold cross-validation required, partial opening, X_train training data, to verify data X_valid in
Assert K>. 1
fold_size = X.shape [ 0] // k # double slash represents addition after their rounded down
X_train, y_train = None, None
for J in Range (K):
IDX = Slice (J * fold_size, (J +. 1) * fold_size) # slice (start, end, step) slice function
X_part, X-y_part = [IDX,:], Y [IDX]
IF J == I:
X_valid in, Y_valid = X_part, y_part
elif X_train None IS:
X_train, y_train = X_part, y_part
the else:
X_train torch.cat = ((X_train, X_part), Dim = 0) increase the number of rows # 0 = Dim, connected vertically
y_train torch.cat = ((y_train, y_part), Dim = 0)
return X_train, y_train, X_valid in, Y_valid
DEFk_fold (k, X_train, y_train, num_epochs, learning_rate, weight_decay, the batch_size):
train_l_sum, valid_l_sum = 0, 0
for I in Range (k):
Data = get_k_fold_data (k, I, X_train, y_train) # acquired k-fold cross-validation the training and validation data
NET = get_net (X_train.shape [. 1]) in this #get_net is a basic linear regression model, implemented method see Appendix. 1
train_ls, valid_ls = train (NET, data *, num_epochs, learning_rate,
weight_decay, batch_size) #train method see below in Appendix 2
train_l_sum train_ls + = [-1] (Range (. 1,. 1 + num_epochs), train_ls, 'epochs', 'RMSE',
+ = valid_ls valid_l_sum [-1]
IF I == 0:
. D2L semilogy
Range (. 1,. 1 + num_epochs), valid_ls,
[ 'Train', 'Valid']) # drawing, and y is a number of requirements, x unchanged. Implemented method see Appendix. 3
Print ( '% D fold, Train RMSE% F, F Valid RMSE%'% (I, train_ls [-1], valid_ls [-1]))
return train_l_sum / K, valid_l_sum / K
* args: acceptance of the parameters of any length, and then stored into one tuple in; as def fun (* args) print ( args), 'fruit', 'animal', 'human' as a parameter in, the output ( 'fruit ',' animal ',' human ')
** kwargs: acceptance of any length parameters, and then stored into a dictionary ; and as
def fun(**kwargs):
for key, value in kwargs.items():
print("%s:%s" % (key,value)
fun (a = 1, b = 2, c = 3) outputs a = 1 b = 2 c = 3
Appendix 1
loss = torch.nn.MSELoss()
def get_net(feature_num):
net = nn.Linear(feature_num, 1)
for param in net.parameters():
nn.init.normal_(param, mean=0, std=0.01)
return net
Appendix 2
def train(net, train_features, train_labels, test_features, test_labels, num_epochs, learning_rate,weight_decay, batch_size):
train_ls, test_ls = [], []
dataset = torch.utils.data.TensorDataset(train_features, train_labels)
train_iter = torch.utils.data.DataLoader(dataset, batch_size, shuffle=True) #TensorDataset和DataLoader的使用请查看我以前的博客
#这里使用了Adam优化算法
optimizer = torch.optim.Adam(params=net.parameters(), lr= learning_rate, weight_decay=weight_decay)
net = net.float()
for epoch in range(num_epochs):
for X, y in train_iter:
l = loss(net(X.float()), y.float())
optimizer.zero_grad()
l.backward()
optimizer.step()
train_ls.append(log_rmse(net, train_features, train_labels))
if test_labels is not None:
test_ls.append(log_rmse(net, test_features, test_labels))
return train_ls, test_ls
附录3
def semilogy(x_vals, y_vals, x_label, y_label, x2_vals=None, y2_vals=None, legend=None, figsize=(3.5, 2.5)):
set_figsize(figsize)
plt.xlabel(x_label)
plt.ylabel(y_label)
plt.semilogy(x_vals, y_vals)
if x2_vals and y2_vals:
plt.semilogy(x2_vals, y2_vals, linestyle=':')
plt.legend(legend)
注：由于最近有其他任务，所以此博客写的匆忙，等我有时间后会丰富，也可能加详细解释。