nn.NNLoss(x,y_target) = -x[y_target]
x=[1,2,3] y_target=[2] 则loss = x[2]=3
nll = nn.NLLLoss()
x = torch.randn(3,3)
y_target = torch.tensor([1,2,0])
cal_my = (x[0][1]+x[1][2] +x[2][0])/3
loss = nll(x,y_target)
print(x)
print(y_target)
print(loss)
print(cal_my)
x为:
tensor([[ 1.5827, 0.2479, -1.3481],
[ 1.6454, -1.1276, 0.3725],
[-0.9897, -1.4498, 2.3366]])
y_target为:
tensor([1, 2, 0])
loss为:
tensor(0.1231)
cal_my为:
tensor(0.1231)
function = nn.LogSoftmax (dim=1)
function(x)代表着先让x经过Softmax,然后再经过torch.log
首先讲一下Softmax的参数设置,在调用Softmax函数时候,需要指定按照哪个维度进行softmax,举例如下:
x = torch.randn(3,3)
softmax_dim0 = nn.Softmax(dim=0)
softmax_dim1 = nn.Softmax(dim=1)
s_0 = softmax_dim0(x)
s_1 = softmax_dim1(x)
print(x)
print('s_0',s_0)
print('s_1',s_1)
输出结果为:
tensor([[ 0.3920, 0.6731, 1.0011],
[-0.0462, 1.1450, -0.5482],
[-0.2710, 1.1763, -1.1713]])
s_0 tensor([[0.4628, 0.2349, 0.7540],
[0.2986, 0.3766, 0.1601],
[0.2385, 0.3885, 0.0859]])
s_1 tensor([[0.2402, 0.3182, 0.4417],
[0.2042, 0.6721, 0.1236],
[0.1767, 0.7514, 0.0718]])
当设置dim=0时候,按照第0维进行softmax 0.4628 + 0.2986+0.2385=1
当设置dim=1时候,按照第1维进行softmax 0.2402+ 0.3182+ 0.4417=1
下面验证nn.LogSoftmax(dim=1)为softmax 和log的组合
softmax = nn.Softmax(dim=1)
logsoftmax = nn.LogSoftmax(dim=1)
x = torch.randn(3,3)
softmax_value = softmax(x)
cal_logsoftmax_value = torch.log(softmax_value)
logsoftmax_value = logsoftmax(x)
print('x',x)
print('softmax_value',softmax_value)
print('cal_logsoftmax_value',cal_logsoftmax_value)
print('logsoftmax_value',logsoftmax_value)
输出结果为:
x tensor([[-0.8330, -0.1150, 0.3054],
[-0.2180, -1.1248, 0.4393],
[-0.4476, 0.7618, 0.8175]])
softmax_value tensor([[0.1620, 0.3322, 0.5058],
[0.3000, 0.1211, 0.5789],
[0.1267, 0.4245, 0.4488]])
cal_logsoftmax_value tensor([[-1.8200, -1.1021, -0.6816],
[-1.2040, -2.1108, -0.5467],
[-2.0662, -0.8568, -0.8011]])
logsoftmax_value tensor([[-1.8200, -1.1021, -0.6816],
[-1.2040, -2.1108, -0.5467],
[-2.0662, -0.8568, -0.8011]])
扫描二维码关注公众号,回复:
10037790 查看本文章
function = nn.CrossEntroyLoss()
function(x)为先让x经过logsoftmax 再经过nlloss,验证如下:
import torch
import torch.nn as nn
cel = nn.CrossEntropyLoss()
nll = nn.NLLLoss()
logsoftmax = nn.LogSoftmax(dim=1)
x = torch.randn(3,3)
y_target = torch.tensor([1,2,0])
logsoftmax_value = logsoftmax(x)
nll_value = nll(logsoftmax_value,y_target)
cel_value = cel(x,y_target)
print('x',x)
print('y_target',y_target)
print('logsoftmax_value',logsoftmax_value)
print('nll_value',nll_value)
print('cel_value',cel_value)
输出结果如下:
x tensor([[ 1.0021, 0.4425, -0.3256],
[ 0.3272, 0.5255, -0.2736],
[ 0.5227, -0.6293, 1.4442]])
y_target tensor([1, 2, 0])
logsoftmax_value tensor([[-0.6079, -1.1675, -1.9355],
[-1.0180, -0.8197, -1.6188],
[-1.3426, -2.4947, -0.4211]])
nll_value tensor(1.3763)
cel_value tensor(1.3763)