1-3 李宏毅2021春季机器学习教程-Google Colab教学-助教许湛然介绍了Colab的使用，这篇文章是助教许湛然关于PyTorch框架的简要讲解。
更多操作查看: https://pytorch.org/docs/stable/tensors.html
目录
Overview of the DNN Training Procedure
torch.nn -- Neural Network Layers
torch.nn -- Activation Functions
torch.nn -- Build your own neural network
Neural Network Evaluation (Validation Set)
Neural Network Evaluation (Testing Set)
Prerequisites-准备工作
熟悉python3的有关知识：if-else, loop等；熟悉numpy，了解数组等操作。
What is PyTorch?-什么是pytorch?
- 开源的机器学习框架
- 提供两个高水平特征的python库
PyTorch v.s. TensorFlow
Overview of the DNN Training Procedure
Tensor
Tensor -- Data Type
ref: torch.Tensor — PyTorch 1.9.1 documentation
Tensor -- Shape of Tensors
Tensor -- Constructor
Tensor -- Operators
squeeze()：主要对数据的维度的进行压缩，去掉维数为1的维度，例如一行或者一列这种，维度为（1,3）的一行三列去掉第一个维数为一的维度之后就变成（3）行。有三种形式：①squeeze(a)就是将a中所有为1的维度删掉，不为1的维度没有影响。②a.squeeze(N) 就是去掉a中指定的维数为一的维度。③还有一种形式就是b=torch.squeeze(a，N)，去掉a中指定的定的维数为一的维度。
unsqueeze()：主要对数据维度进行填充。给指定位置加上维数为一的维度，例如有个三行的数据（3），在0的位置加了一维就变成一行三列（1,3）。
transpose()：交换矩阵的两个维度，transpose(dim0, dim1) → Tensor，其和torch.transpose()函数作用一样。
cat()：拼接函数。在给定维度上对输入的张量序列seq 进行连接操作。torch.cat()可以看做 torch.split() 和 torch.chunk()的反操作。
more operators: torch.Tensor — PyTorch 1.9.1 documentation
Tensor -- PyTorch v.s. NumPy
ref: https://github.com/wkentaro/pytorch-for-numpy-users
Tensor -- Device
Tensor -- Device(GPU)
上图的链接如下：
https://towardsdatascience.com/what-is-a-gpu-and-do-you-need-one-in-deep-learning-718b9597aa0d
How to Calculate Gradient?
Load Data
Dataset & Dataloader
注意shuffle参数，在train时为True，在test时为False：
Define Neural Network
torch.nn -- Neural Network Layers
矩阵与向量表示：
代码如下：
torch.nn -- Activation Functions
Loss Function
torch.nn -- Loss Functions
- Mean Squared Error (for linear regression)回归
nn.MSELoss()
- Cross Entropy (for classification)分类
nn.CrossEntropyLoss()
torch.nn -- Build your own neural network
代码：
import torch.nn as nn
class MyModel(nn.Module):
def __init__(self):
super(MyModel, self).__init__()
self.net = nn.Sequential(
nn.Linear(10, 32),
nn.Sigmoid(),
nn.Linear(32, 1)
)
def forward(self, x):
return self.net(x)Optimizer
torch.optim
代码：
torch.optim.SGD(params,lr,momentum = 0)Neural Network Training
前期准备
代码如下：
dataset = MyDataset(file)
tr_set = DataLoader(dataset, 16, shuffle=True)
model = MyModel().to(device)
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), 0.1)多次epoch
代码如下：
for epoch in range(n_epochs):
model.train()
for x, y in tr_set:
optimizer.zero_grad()
x, y = x.to(device), y.to(device)
pred = model(x)
loss = criterion(pred, y)
loss.backward()
optimizer.step()Neural Network Evaluation (Validation Set)
代码如下：
model.eval()
total_loss = 0
for x, y in dv_set:
x, y = x.to(device), y.to(device)
with torch.no_grad():#不希望进行梯度计算
pred = model(x)
loss = criterion(pred, y)
total_loss += loss.cpu().item() * len(x)
avg_loss = total_loss / len(dv_set.dataset)Neural Network Evaluation (Testing Set)
代码如下：
model.eval()
preds = []
for x in tt_set:
x = x.to(device)
with torch.no_grad():
pred = model(x)
preds.append(pred.cpu())Save/Load a Neural Network
代码如下：
#Save
torch.save(model.state_dict(), path)
# Load
ckpt = torch.load(path)
model.load_state_dict(ckpt)More About PyTorch
- torchaudio
speech/audio processing
- torchtext
natural language processing
- torchvision
computer vision
- skorch
scikit-learn + pyTorch
- Useful github repositories using PyTorch
- Huggingface Transformers (transformer models: BERT, GPT, ...)
- Fairseq (sequence modeling for NLP & speech)
- ESPnet (speech recognition, translation, synthesis, ...)
- Many implementation of papers
- ...
Reference
GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Pytorch vs. TensorFlow: What You Need to Know | Udacity
说明：记录学习笔记，如果错误欢迎指正！写文章不易，转载请联系我。