AutoEncoder (from the encoder - unsupervised learning)
neural network can be carried out unsupervised learning, training data only, no tag data from the encoder is such a form.
Since coding can automatically classify data, but can also be nested the above semi-supervised learning, a small amount of labeled samples and a large number of unlabeled samples to learn.
Import Torch Import torch.nn AS NN Import torch.utils.data AS the Data Import torchvision Import matplotlib.pyplot AS PLT from mpl_toolkits.mplot3d Import Axes3D from matplotlib Import cm & lt Import numpy NP AS # hyperparametric # the Hyper the Parameters the EPOCH = 10 BATCH_SIZE = 64 the LR = 0.005 # Learning Rate DOWNLOAD_MNIST = True # False # through the data, it may be arranged False N_TEST_IMG. 5 = # 到时候显示5张图片看效果 # 下载数据 # Mnist digits dataset train_data = torchvision.datasets.MNIST( root='./mnist/', train=True, # this is training data transform=torchvision.transforms.ToTensor(), # Converts a PIL.Image or numpy.ndarray to # torch.FloatTensor of shape (C x H x W) and normalize in the range [0.0, 1.0] download=DOWNLOAD_MNIST, # download it if you don't have it ) # plot one example print(train_data.train_data.size()) # (60000, 28, 28) print(train_data.train_labels.size()) # (60000) plt.imshow(train_data.train_data[2].numpy(), cmap='gray') plt.title('%i' % train_data.train_labels[2]) plt.show() # 加载训练数据 # Data Loader for easy mini-batch return in training, the image batch shape will be (50, 1, 28, 28) train_loader = Data.DataLoader(dataset=train_data, batch_size=BATCH_SIZE, shuffle=True) # AutoEncoder #AutoEncoder very simple form, namely Decoder and encoder, compression and decompression, the compressed feature values obtained after compression, and then extract the values from the original image into a compressed feature. Class AutoEncoder (nn.Module): DEF the __init__ (Self): Super ( AutoEncoder, Self). the __init__ () # compressed self.encoder = nn.Sequential ( nn.Linear ( 28 * 28, 128 ), nn.Tanh (), nn.Linear ( 128, 64 ), nn.Tanh (), nn.Linear ( 64, 12 is ), nn.Tanh (), nn.Linear ( 12 is,. 3), #compress to 3 features which can be visualized in plt # 3 wherein compressed into, for visualization of 3D image ) # decompression self.decoder = nn.Sequential ( nn.Linear ( . 3, 12 is ), nn.Tanh (), nn.Linear ( 12 is, 64 ), nn.Tanh (), nn.Linear ( 64, 128 ), nn.Tanh (), nn.Linear ( 128, 28 * 28 ), nn.Sigmoid (), # the compress to A Range ( 0, 1) # excitation function so that an output value (0, 1) ) DEF Forward (Self, X): encoded = self.encoder(x) decoded = self.decoder(encoded) return encoded, decoded autoencoder = AutoEncoder() optimizer = torch.optim.Adam(autoencoder.parameters(), lr=LR) loss_func = nn.MSELoss() # initialize figure f, a = plt.subplots(2, N_TEST_IMG, figsize=(5, 2)) plt.ion() # continuously plot # original data (first row) for viewing view_data = train_data.train_data[:N_TEST_IMG].view(-1, 28*28).type(torch.FloatTensor)/255. for i in range(N_TEST_IMG): A [0] [I] .imshow (np.reshape (view_data.data.numpy () [I], ( 28, 28)), CMap = ' Gray ' ); A [0] [I] .set_xticks (( )); a [0] [i] .set_yticks (()) # training # can effectively use encoder and decoder to do a lot of things, such as where we see and compare the original image with the information output of the decoder, # can use . after compression encoder point of view, understanding the neural network of the original picture encoder can approximate different picture data separated. # this is a unsupervised learning process. for Epoch in the Range (EPOCH): for the STEP, (the X- , b_label) in the enumerate (train_loader): B_X = x.view (-1, 28 * 28) # BATCH X, Shape (BATCH, 28 * 28) b_y = x.view (-1, 28 * 28) # batch y, shape (batch, 28*28) encoded, decoded = autoencoder(b_x) loss = loss_func(decoded, b_y) # mean square error optimizer.zero_grad() # clear gradients for this training step loss.backward() # backpropagation, compute gradients optimizer.step() # apply gradients if step % 100 == 0: print('Epoch: ', epoch, '| train loss: %.4f' % loss.data.numpy()) # plotting decoded image (second row) _, decoded_data = autoencoder(view_data) for i in range(N_TEST_IMG): a[1][i].clear() a[1][i].imshow(np.reshape(decoded_data.data.numpy()[i], (28, 28)), cmap='gray') a[1][i].set_xticks(()); a[1][i].set_yticks(()) plt.draw(); plt.pause(0.05) plt.ioff() plt.show() # 画3D图 # visualize in 3D plot # 要观看的数据 = train_data.train_data view_data [: 200 is] .view (-1, 28 * 28) .Type (torch.FloatTensor) / 255 . encoded_data, _ = autoencoder (view_data) # extract the compressed feature values fig = plt.figure (2 ) AX = Axes3D (Fig) # 3D FIGS # X, Y, Z value data X, Y, Z = encoded_data.data [ :, 0] .numpy (), encoded_data.data [:, 1] .numpy () , encoded_data.data [:, 2 ] .numpy () values = train_data.train_labels [: 200 is] .numpy () # tag value for X, Y, Z, S in ZIP (X-, the Y, the Z, values): C cm.rainbow = (int (* 255 S /. 9)) # color ax.text(x, y, z, s, backgroundcolor=c) # 标位子 ax.set_xlim(X.min(), X.max()); ax.set_ylim(Y.min(), Y.max()); ax.set_zlim(Z.min(), Z.max()) plt.show()
Sometimes the neural network to accept a large number of input information, such as high-definition picture input information is input information may reach tens of millions, so that the neural network to learn directly from tens of millions of information sources is a very difficult job.
So, why not compress, extract the most representative of the original picture information, reduce the amount of input information, then the information after the cut into the neural network learning so much easier learning curve easier.
Therefore, since the encoding can play a role in this case by an X-white original data compression and decompression in black X, X and then by black and white contrast, obtains a prediction error, reverse transfer, and gradually improve the accuracy of the self-encoded .
Trained since the middle of this part of the coding is able to sum up the essence of the original data can be seen, from start to finish, we only used the input data X, and X did not use the data corresponding to the tag, so it can be said is a self-encoded kind of unsupervised learning.
The time to actually use a self-coded usually only use the first half of the self coding.
