Article directory
3. Numpy and Tensor
3. Index of Tensor
(1) item: If the Tensor is a single element, return a scalar, otherwise an error occurs
a = torch.randn(1)
print(a)
>>>tensor([-0.0288])
print(a.item())
>>>-0.028787262737751007
(2) index_select(input,dim,index): Select some rows or columns on the specified dimension
The following introduces two calling methods
. dim represents the dimension, 1 and -1 in two dimensions can represent columns, and 2 and -1 in three dimensions can represent columns, and
index represents the returned index or serial number.
In the following example, (1, 1) represents the second value of the second dimension of the index, and the second dimension represents a column, so it is the second column.
import torch
torch.manual_seed(100)
x = torch.randn(2,3)
print(x)
>>>tensor([[ 0.3607, -0.2859, -0.3938],
[ 0.2429, -1.3833, -2.3134]])
print(x.index_select(1,torch.LongTensor([1])))
>>>tensor([[-0.2859],
[-1.3833]])
print(torch.index_select(x,1,torch.LongTensor([1])))
>>>tensor([[-0.2859],
[-1.3833]])
(3) nonzero(input): Get the subscript of non-zero elements
import torch
x = torch.tensor([[2,3,0],[0,3,1],[1,0,3]])
print(x)
>>>tensor([[2, 3, 0],
[0, 3, 1],
[1, 0, 3]])
print(x.nonzero())
>>>tensor([[0, 0],
[0, 1],
[1, 1],
[1, 2],
[2, 0],
[2, 2]])
print(torch.nonzero(x))
>>>tensor([[0, 0],
[0, 1],
[1, 1],
[1, 2],
[2, 0],
[2, 2]])
(4) masked_select(input,mask): use a binary value to select
import torch
torch.manual_seed(100)
x = torch.randn(2,3)
y = torch.randn(1,2)
print(x)
>>>tensor([[ 0.3607, -0.2859, -0.3938],
[ 0.2429, -1.3833, -2.3134]])
# 类似Numpy的索引方式
print(x[0,:])
>>>tensor([ 0.3607, -0.2859, -0.3938])
print(x[:,-1])
>>>tensor([-0.3938, -2.3134])
# 生成是否大于0的张量
mask = x>0
# 输出mask中True对应的元素
print(torch.masked_select(x,mask))
>>>tensor([0.3607, 0.2429])
# 取出1对应的元素
mask = torch.tensor([[1,1,0],[0,0,1]],dtype=torch.bool)
print(torch.masked_select(x,mask))
>>>tensor([ 0.3607, -0.2859, -2.3134])
# 取出1，2列元素
mask = torch.tensor([1,1,0],dtype=torch.bool)
print(torch.masked_select(x,mask))
>>>tensor([ 0.3607, -0.2859, 0.2429, -1.3833])
# 取出第2行元素
mask = torch.tensor([[0],[1]],dtype=torch.bool)
print(torch.masked_select(x,mask))
>>>tensor([ 0.2429, -1.3833, -2.3134])
(5) gather(input,dim,index): Select data in the specified dimension, and the output shape is consistent with index
Dim is the dimension. For example, the value range of two dimensions is [-2, 1] as
the following example, 0, -2 means the first dimension, 1, -1 means the second dimension
x.gather(1, y) according to the first dimension , returns the element at that position in x indicated by the corresponding element of y.
As shown in the above two pictures:
when dim=1:
when y is all 0, so each column returned is the value corresponding to the 0th column in x
When the first element of y=1, return (0, 0) The value of the position is the value of the 0th row of the first column in x.
As shown in the above two pictures:
when dim=0:
when y is all 0, so each row returned is the value corresponding to the 0th row in x.
When the yth row When an element = 1, the returned value at position (0, 0) is the value of row 1, column 0 in x
import torch
x = torch.tensor([[2,3,0],[0,3,1],[1,0,3]])
y = torch.zeros_like(x)
print(x)
>>>tensor([[2, 3, 0],
[0, 3, 1],
[1, 0, 3]])
print(x.gather(1,y))
>>>tensor([[2, 2, 2],
[0, 0, 0],
[1, 1, 1]])
print(torch.gather(x,0,y))
>>>tensor([[2, 3, 0],
[2, 3, 0],
[2, 3, 0]])
print(torch.gather(x,-2,y))
>>>tensor([[2, 3, 0],
[2, 3, 0],
[2, 3, 0]])
y[0,0] = 1
print(x)
>>>tensor([[2, 3, 0],
[0, 3, 1],
[1, 0, 3]])
print(x.gather(1,y))
>>>tensor([[3, 2, 2],
[0, 0, 0],
[1, 1, 1]])
print(torch.gather(x,0,y))
>>>tensor([[0, 3, 0],
[2, 3, 0],
[2, 3, 0]])
print(torch.gather(x,-2,y))
>>>tensor([[0, 3, 0],
[2, 3, 0],
[2, 3, 0]])
(6) scatter_(input,dim,index,src): It is the reverse operation of gather, supplementing data according to the specified index
The following example:
index has only 2 lines, so only the first two lines of x are reassigned.
When dim=0,
x[index[0][0]][0] = y[0][0] ie x[1][0] = 1
x[index[1][1]][1] = y[1][1] ie x[0][1] = 5
x[index[1][2]][2] = y[1][2] ie x[0][2] = 6
x[index[1][0]][0] = y[1][0] ie x[0][0] = 4
Note: The index of y is consistent with the index index
When dim=1,
x[0][index[0][2]] = y[0][2] i.e. x[0][0] = 3
x[0][index[0][0]] = y[0][0] i.e. x[0][1] = 1
x[1][index[1][2]] = y[1][2] i.e. x[1][0] = 6
import torch
x = torch.tensor([[2,3,0],[0,3,1],[1,0,3]])
y = torch.tensor([[1,2,3],[4,5,6]])
index = torch.tensor([[1,0,0],[0,0,0]])
print(x)
>>>tensor([[2, 3, 0],
[0, 3, 1],
[1, 0, 3]])
print(y)
>>>tensor([[1, 2, 3],
[4, 5, 6]])
print(x.scatter_(0,index,y))
>>>tensor([[4, 5, 6],
[1, 3, 1],
[1, 0, 3]])
print(x.scatter_(1,index,y))
>>>tensor([[3, 1, 6],
[6, 3, 1],
[1, 0, 3]])
4. Tensor broadcast mechanism
import torch
import numpy as np
A = np.arange(0, 40, 10).reshape(4, 1)
B = np.arange(0, 3)
# 把ndarray转换为tensor
A1 = torch.from_numpy(A)
B1 = torch.from_numpy(B)
# tensor自动实现广播
C = A1 + B1
print(A1)
>>>tensor([[ 0],
[10],
[20],
[30]], dtype=torch.int32)
print(B1)
>>>tensor([0, 1, 2], dtype=torch.int32)
print(C)
>>>tensor([[ 0, 1, 2],
[10, 11, 12],
[20, 21, 22],
[30, 31, 32]], dtype=torch.int32)
# 手工配置
# 根据规则1，B1需要向A1看齐，把B变为（1，3）
B2 = B1.unsqueeze(0)
# 重复数组，分别为（4，3）矩阵
A2 = A1.expand(4,3)
B3 = B2.expand(4,3)
# 相加
C1 = A2 + B3
print(A2)
>>>tensor([[ 0, 0, 0],
[10, 10, 10],
[20, 20, 20],
[30, 30, 30]], dtype=torch.int32)
print(B3)
>>>tensor([[0, 1, 2],
[0, 1, 2],
[0, 1, 2],
[0, 1, 2]], dtype=torch.int32)
print(C1)
>>>tensor([[ 0, 1, 2],
[10, 11, 12],
[20, 21, 22],
[30, 31, 32]], dtype=torch.int32)
5. Element-by-element operations
(1) Absolute value abs | addition add
import torch
t = torch.randn(1,3)
t1 = torch.randn(1,3)
print(t)
>>>tensor([[ 2.0261, -0.9644, 1.0573]])
print(t1)
>>>tensor([[ 1.0033, -0.1407, 0.4890]])
print(t.abs())
>>>tensor([[2.0261, 0.9644, 1.0573]])
print(t.add(t1))
>>>tensor([[ 3.0294, -1.1052, 1.5462]])
(2)addcdiv(t,v,t1,t2)
t + 100*(t1/t2)
import torch
t = torch.randn(1,3)
t1 = torch.randn(1,3)
t2 = torch.randn(1,3)
print(t)
>>>tensor([[-0.5238, -0.3393, -0.7081]])
print(t1)
>>>tensor([[-0.5984, 1.5229, 0.7923]])
print(t2)
>>>tensor([[ 1.5784, -1.4862, -1.5006]])
# t + 100*(t1/t2)
print(torch.addcdiv(t,100,t1,t2))
>>>tensor([[ -38.4344, -102.8099, -53.5090]])
(3)addcmul(t,v,t1,t2)
t + 100*(t1*t2)
import torch
t = torch.randn(1,3)
t1 = torch.randn(1,3)
t2 = torch.randn(1,3)
print(t)
>>>tensor([[-0.1856, 1.8894, 1.1288]])
print(t1)
>>>tensor([[-0.6742, 2.0252, -0.6274]])
print(t2)
>>>tensor([[-1.7973, -1.3763, 1.0582]])
# t + 100*(t1*t2)
print(torch.addcmul(t,100,t1,t2))
>>>tensor([[ 120.9984, -276.8339, -65.2656]])
(4) round up ceil | round down floor
import torch
t = torch.randn(1,3)
print(t)
print(t.ceil())
print(t.floor())
>>>tensor([[-0.8639, -0.6445, 2.4636]])
tensor([[-0., -0., 3.]])
tensor([[-1., -1., 2.]])
(5) Limit the tensor to the specified interval
import torch
t = torch.randn(1,3)
print(t)
print(t.clamp(0,1))
>>>tensor([[ 0.1364, 0.8311, -0.1269]])
tensor([[0.1364, 0.8311, 0.0000]])
(6) exponent exp | logarithm log | power pow
import torch
t = torch.tensor([1,4,9])
print(t)
print(t.exp())
print(t.log())
print(t.pow(2))
>>>tensor([1, 4, 9])
tensor([2.7183e+00, 5.4598e+01, 8.1031e+03])
tensor([0.0000, 1.3863, 2.1972])
tensor([ 1, 16, 81])
(7) Element-by-element multiplication mul* | Negative neg
import torch
t = torch.tensor([2,4,8])
print(t)
print(t.mul(2))
print(t*2)
print(t.neg())
>>>tensor([2, 4, 8])
tensor([ 4, 8, 16])
tensor([ 4, 8, 16])
tensor([-2, -4, -8])
(8) Activation function sigmoid / tanh / softmax
import torch
t = torch.tensor([2.,4.,8.])
print(t)
print(t.sigmoid())
print(t.tanh())
print(t.softmax(0))
>>>tensor([2., 4., 8.])
tensor([0.8808, 0.9820, 0.9997])
tensor([0.9640, 0.9993, 1.0000])
tensor([0.0024, 0.0179, 0.9796])
(9) Take the sign sign | open the square root sign sqrt
import torch
t = torch.tensor([2,-4,8])
print(t)
print(t.sign())
print(t.sqrt())
>>>tensor([ 2, -4, 8])
tensor([ 1, -1, 1])
tensor([1.4142, nan, 2.8284])
6. Merge operation
(1)cumprod(t,axis) accumulates t in the specified dimension
import torch
t = torch.linspace(0,10,6)
t = t.view((2,3))
print(t)
print(t.cumprod(0))
print(t.cumprod(1))
>>>tensor([[ 0., 2., 4.],
[ 6., 8., 10.]])
tensor([[ 0., 2., 4.],
[ 0., 16., 40.]])
tensor([[ 0., 0., 0.],
[ 6., 48., 480.]])
(2) cumsum(t,axis) accumulates t in the specified dimension
import torch
t = torch.linspace(0,10,6)
t = t.view((2,3))
print(t)
print(t.cumsum(0))
print(t.cumsum(1))
>>>tensor([[ 0., 2., 4.],
[ 6., 8., 10.]])
tensor([[ 0., 2., 4.],
[ 6., 10., 14.]])
tensor([[ 0., 2., 6.],
[ 6., 14., 24.]])
(3) dist(a,b,p=2) returns the p-order norm between a and b
import torch
t = torch.randn(2,3)
t1 = torch.randn(2,3)
print(t)
print(t1)
print(torch.dist(t,t1,2))
>>>tensor([[ 0.7374, 0.3365, -0.3164],
[ 0.9939, 0.9986, -1.8464]])
tensor([[-0.4015, 0.7628, 1.5155],
[-1.9203, -1.0317, -0.0433]])
tensor(4.5498)
(4) mean\median mean\median
import torch
t = torch.randn(2,3)
print(t)
print(t.mean())
print(t.median())
>>>tensor([[-1.2806, -0.2002, -1.6772],
[-1.0748, 0.9528, -0.9231]])
tensor(-0.7005)
tensor(-1.0748)
(5)std\var standard deviation\variance
import torch
t = torch.randn(2,3)
print(t)
print(t.std())
print(t.var())
>>>tensor([[-0.3008, 0.5051, -0.3573],
[ 0.6127, 0.5265, -0.2748]])
tensor(0.4727)
tensor(0.2234)
(6) norm(t,p=2) returns the p-order norm of t
import torch
t = torch.randn(2,3)
print(t)
print(t.norm(2))
>>>tensor([[-1.4880, -0.6627, 0.0032],
[-2.0794, -0.4204, 0.7097]])
tensor(2.7673)
(7)prod(t) \ sum(t) returns the product\sum of all elements of t
import torch
t = torch.randn(2,3)
print(t)
print(t.prod())
print(t.sum())
>>>tensor([[ 1.0360, 1.5088, 2.3022],
[-0.9142, 1.9266, -0.8202]])
tensor(5.1986)
tensor(5.0393)
7. Comparison operation
(1) eq compares whether Tensor is equal, supports broadcast
import torch
t = torch.tensor([[1,2,3],[4,5,6]])
t1 = torch.tensor([[1,2,3],[4,5,6]])
print(t)
print(t1)
print(t.eq(t1))
>>>tensor([[1, 2, 3],
[4, 5, 6]])
tensor([[1, 2, 3],
[4, 5, 6]])
tensor([[True, True, True],
[True, True, True]])
(2) equal compares whether Tensor has the same shape and value
import torch
t = torch.tensor([[1,2,3],[4,5,6]])
t1 = torch.tensor([[1,2,3],[4,5,6]])
t2 = torch.tensor([1,2,3,4,5,6])
print(t)
print(t1)
print(t2)
print(t.equal(t1))
print(t.equal(t2))
>>>tensor([[1, 2, 3],
[4, 5, 6]])
tensor([[1, 2, 3],
[4, 5, 6]])
tensor([1, 2, 3, 4, 5, 6])
True
False
(3)ge\le\gt\lt greater than\less than\greater than or equal to\less than or equal to
import torch
t = torch.tensor([[1,2,3],[4,5,6]])
t1 = torch.tensor([[3,2,1],[6,5,4]])
print(t)
print(t1)
>>>tensor([[1, 2, 3],
[4, 5, 6]])
tensor([[3, 2, 1],
[6, 5, 4]])
print(t.ge(t1))
>>>tensor([[False, True, True],
[False, True, True]])
print(t.le(t1))
>>>tensor([[ True, True, False],
[ True, True, False]])
print(t.gt(t1))
>>>tensor([[False, False, True],
[False, False, True]])
print(t.lt(t1))
>>>tensor([[ True, False, False],
[ True, False, False]])
(4)max\min(t,axis) returns the maximum value, if axis is specified, additional subscripts will be returned
import torch
t = torch.tensor([[1,2,3],[4,5,6]])
print(t)
>>>tensor([[1, 2, 3],
[4, 5, 6]])
print(t.max())
>>>tensor(6)
print(t.max(1))
>>>torch.return_types.max(
values=tensor([3, 6]),
indices=tensor([2, 2]))
print(t.min())
>>>tensor(1)
print(t.min(1))
>>>torch.return_types.min(
values=tensor([1, 4]),
indices=tensor([0, 0]))
(5)topk(t,k,axis) takes the highest K values on the specified axis dimension
import torch
t = torch.tensor([[1,2,3],[4,5,6]])
print(t)
>>>tensor([[1, 2, 3],
[4, 5, 6]])
print(t.topk(1,0))
>>>torch.return_types.topk(
values=tensor([[4, 5, 6]]),
indices=tensor([[1, 1, 1]]))
print(t.topk(1,1))
>>>torch.return_types.topk(
values=tensor([[3],
[6]]),
indices=tensor([[2],
[2]]))
print(t.topk(2,0))
>>>torch.return_types.topk(
values=tensor([[4, 5, 6],
[1, 2, 3]]),
indices=tensor([[1, 1, 1],
[0, 0, 0]]))
print(t.topk(2,1))
>>>torch.return_types.topk(
values=tensor([[3, 2],
[6, 5]]),
indices=tensor([[2, 1],
[2, 1]]))
8. Matrix operation
(1)dot(t1,t2) calculates the inner product/dot product of tensor (1D)
import torch
t = torch.tensor([3,4])
t1 = torch.tensor([2,3])
print(t)
print(t1)
print(t.dot(t1))
>>>tensor([3, 4])
tensor([2, 3])
tensor(18)
(2)mm(mat1,mat2) \ bmm(batch1,batch2) Calculate matrix multiplication/3D matrix multiplication with batch
import torch
t = torch.randint(10,(2,3))
t1 = torch.randint(6,(3,4))
print(t)
print(t1)
print(t.mm(t1))
>>>tensor([[9, 3, 9],
[0, 3, 8]])
tensor([[3, 0, 0, 1],
[4, 5, 5, 3],
[2, 0, 1, 4]])
tensor([[57, 15, 24, 54],
[28, 15, 23, 41]])
t = torch.randint(10,(2,2,3))
t1 = torch.randint(6,(2,3,4))
print(t)
print(t1)
print(t.bmm(t1))
>>>tensor([[[7, 4, 0],
[5, 5, 1]],
[[1, 4, 1],
[3, 3, 6]]])
tensor([[[0, 3, 2, 5],
[3, 2, 1, 3],
[3, 3, 4, 5]],
[[3, 5, 2, 3],
[2, 1, 0, 5],
[5, 1, 2, 2]]])
tensor([[[12, 29, 18, 47],
[18, 28, 19, 45]],
[[16, 10, 4, 25],
[45, 24, 18, 36]]])
(3)mv(t1,v1) calculates matrix and vector multiplication
import torch
t = torch.randint(10,(2,2))
t1 = torch.tensor([1,2])
print(t)
print(t1)
print(t.mv(t1))
>>>tensor([[0, 7],
[5, 5]])
tensor([1, 2])
tensor([14, 15])
(4)t transpose
import torch
t = torch.randint(10,(2,2))
print(t)
print(t.t())
>>>tensor([[5, 3],
[0, 8]])
tensor([[5, 0],
[3, 8]])
(5)svd(t) Calculate the SVD decomposition of t
import torch
t = torch.tensor([[1.,2.],[3.,4.]])
print(t)
>>>tensor([[1., 2.],
[3., 4.]])
print(t.svd())
>>>torch.return_types.svd(
U=tensor([[-0.4046, -0.9145],
[-0.9145, 0.4046]]),
S=tensor([5.4650, 0.3660]),
V=tensor([[-0.5760, 0.8174],
[-0.8174, -0.5760]]))
9. Comparison of PyTorch and Numpy
| Operation category | Numpy | PyTorch |
|---|---|---|
| type of data | np.ndarray | torch.Tensor |
| np.float32 | torch.float32;torch.float | |
| np.float64 | torch.float64;torch.double | |
| np.int64 | torch.int64;torch.long | |
| build from existing data | np.arrat([1,2],dtype=np.float16) | torch.tensor([1,2],dtype.float16) |
| x.copy() | x.clone() | |
| np.concatenate | torch.cat | |
| Linear Algebra | np.dot | torch.mm |
| Attributes | x | x.dim |
| x.size | x.nelement | |
| shape manipulation | x.reshape | x.reshape;x.view |
| x.flatten | x.view(-1) | |
| type conversion | np.floor(x) | torch.floor(x);x.floor() |
| Compare | np.less | x.lt |
| np.less_equal ; np.greater | x.le / x.gt | |
| np.greate_equal / np.equal / np.not_equal | x.ge / x.eq / x.ne | |
| random seed | np.random.seed | torch.manual_seed |
related suggestion
[pyTorch Study Notes①] Numpy Basics Part 1
[pyTorch Study Notes ②] Numpy Basics Part 2
[pyTorch Study Notes ③] PyTorch Basics Part 1