Article directory
1. Introduction to PyTorch
PyTorch is a deep learning framework released by Facebook. It inherits Torch's flexible and dynamic programming environment and user-friendly interface. It supports the construction of dynamic neural networks in a fast and flexible manner, and also allows rapid code changes during training without Hindering its performance, its ability to support cutting-edge AI models such as motion graphics is ideal for quick experiments.
It mainly consists of 4 main packages:
(1) torch: can convert the tensor type to torch.cuda.TensorFloat, and perform calculations on the GPU.
(2) torch.autograd: a package for building computational graphs and automatically obtaining gradients
(3) torch.nn: a neural network library with shared layers and loss functions
(4) torch.optim: with general optimization algorithms (such as SGD, Adam et al.) optimization package
2. Installation of PyTorch
It is best to install an anaconda or miniconda first, then create a virtual environment, and then install PyTorch
https://pytorch.org/get-started/locally/
1. CPU version
2. GPU version
For the GPU version, the GPU driver, CUDA, and cuDNN computing framework must be installed first.
See: https://mp.csdn.net/mp_blog/creation/editor/120840528
and then install PyTorch:
3. Numpy and Tensor
PyTorch's Tensor can be zero-dimensional, one-dimensional, or multi-dimensional. It is similar to Numpy. The two can share memory, and the conversion is convenient and efficient. The difference between the two is that Numpy can only be calculated on the CPU, while Tensor can choose to run on the GPU in the GPU environment.
From the interface, it can be divided into two categories:
(1) torch.function. Such as torch.sum, torch.add, etc.
(2) tensor.function. Such as tensor.view, tensor.add, etc.
But the two are mostly equivalent. For example, x.add(y) is equivalent to torch.add(x,y).
Whether to modify its own data can also be divided into two categories:
(1) Do not modify its own data: such as x.add(y), after the end of the operation, the value of x remains unchanged, and return a new tensor
(2) Modify its own data: such as x.add_(y), the operation result is saved in x, that is, x is modified.
import torch
x = torch.tensor([1,2])
y = torch.tensor([3,4])
z = x.add(y)
print(x)
>>>tensor([1, 2])
print(z)
>>>tensor([4, 6])
x.add_(y)
print(x)
>>>tensor([4, 6])
1. Tensor creation
Two notes:
(1) torch.Tensor is a mixture of torch.tensor and torch.empty, but when data is passed in, torch.Tensor uses the global default dtype (FloatTensor), while torch.tensor infers data from the data type.
(2) torch.tensor(1) returns a fixed value of 1, while torch.Tensor(1) returns a tensor of size 1, and the value is random.
t1 = torch.Tensor(1)
print(t1)
>>>tensor([-1.0713])
t2 = torch.tensor(1)
print(t2)
>>>tensor(1)
(1) Tensor: Constructed directly from parameters
print(torch.Tensor([1,2,3]))
>>>tensor([1., 2., 3.])
print(torch.Tensor([[1,2],[3,4]]))
>>>tensor([[1., 2.],
[3., 4.]])
(2) Tensor/eye: Specified shape generation
# 指定形状，随机生成
print(torch.Tensor(2,3))
>>>tensor([[-3.5458e-16, -1.4412e+17, -3.5458e-16],
[ 4.2352e-22, -3.5457e-16, 5.6296e+14]])
print(torch.eye(2,3))
>>>tensor([[1., 0., 0.],
[0., 1., 0.]])
(3) linspace: divide evenly in a given range
print(torch.linspace(1,10,10))
>>>tensor([ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10.])
(4) logspace: the given range is equally divided (different from 3, the range is based on 10)
print(torch.logspace(1,2,10))
>>>tensor([ 10.0000, 12.9155, 16.6810, 21.5443, 27.8256, 35.9381, 46.4159,
59.9484, 77.4264, 100.0000])
(5) rand: specified shape, [0, 1) uniform distribution
print(torch.rand(2,3))
>>>tensor([[0.4956, 0.5236, 0.1909],
[0.5445, 0.1571, 0.3773]])
(6) randn: specified shape, standard normal distribution
print(torch.randn(2,3))
>>>tensor([[-0.6656, -0.0725, 1.9198],
[ 0.6756, -1.8594, -0.8263]])
(7) ones: specify the shape, the initial value is 1
print(torch.ones(2,3))
>>>tensor([[1., 1., 1.],
[1., 1., 1.]])
(8) zeros: specify the shape, the initial value is 0
print(torch.zeros(2,3))
>>>tensor([[0., 0., 0.],
[0., 0., 0.]])
(9) ones_like: imitate the shape, the initial value is 1
a = torch.Tensor(2,3)
print(torch.ones_like(a))
>>>tensor([[1., 1., 1.],
[1., 1., 1.]])
(10) zeros_like: imitate the shape, the initial value is 0
a = torch.Tensor(2,3)
print(torch.zeros_like(a))
>>>tensor([[0., 0., 0.],
[0., 0., 0.]])
(11) arange: In the specified interval, generate a sequence tensor at the specified interval
print(torch.arange(1,10,1))
>>>print(torch.arange(1,10,1))
(12) from_Numpy: Create a Tensor from ndarray
a = np.arange(1,10,1)
print(a)
>>>[1 2 3 4 5 6 7 8 9]
print(torch.from_numpy(a))
>>>tensor([1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=torch.int32)
2. Deformation of Tensor
(1) View the shape
a = torch.randn(2,3)
print(a.size())
>>>torch.Size([2, 3])
print(a.shape)
>>>torch.Size([2, 3])
# 返回a的维度
print(a.dim())
>>>2
(2) Calculate the number of elements
a = torch.randn(2,3)
print(torch.numel(a))
>>>6
(3) Modify the shape
a = torch.randn(2,3)
print(a)
>>>tensor([[-0.0169, 0.3507, -0.8997],
[ 0.1978, -1.3577, 0.7491]])
print(a.view(3,2))
>>>tensor([[-0.0169, 0.3507],
[-0.8997, 0.1978],
[-1.3577, 0.7491]])
print(a) # 可见，view没有修改a自身
>>>tensor([[-0.0169, 0.3507, -0.8997],
[ 0.1978, -1.3577, 0.7491]])
print(a.resize(2,3))
>>>UserWarning: non-inplace resize is deprecated
warnings.warn("non-inplace resize is deprecated")
tensor([[-0.0169, 0.3507, -0.8997],
[ 0.1978, -1.3577, 0.7491]])
print(a) # 可见，resize修改了a自身，官方也不建议这样修改。
>>>tensor([[-0.0169, 0.3507, -0.8997],
[ 0.1978, -1.3577, 0.7491]])
a = torch.randn(2,3)
print(a.reshape(3,2))
>>>tensor([[-2.1509, 1.9620],
[-0.3320, -1.8106],
[-2.1936, 0.1626]])
print(a)
>>>tensor([[-2.1509, 1.9620, -0.3320],
[-1.8106, -2.1936, 0.1626]])
# 增加维度
a = torch.randn(2,3)
print(a)
>>>tensor([[ 0.3440, -0.9412, 1.0980],
[-1.2131, 1.1445, 2.2529]])
z = torch.unsqueeze(a,0)
print(z)
>>>tensor([[[ 0.3440, -0.9412, 1.0980],
[-1.2131, 1.1445, 2.2529]]])
# 压缩维度
a = torch.randn(2,1,3,1)
print(a.size())
>>>torch.Size([2, 1, 3, 1])
z = torch.squeeze(a)
print(z.size()) # 将全部大小为1的维压缩
>>>torch.Size([2, 3])
z = torch.squeeze(a,3)
print(z.size())# 若指定维的大小为1则压缩该维，不为1不压缩
>>>torch.Size([2, 1, 3])
related suggestion
[pyTorch Study Notes ①] Numpy Basics Part 1
[pyTorch Study Notes ②] Numpy Basics Part 2