nn.CrossEntroyLoss nn.NLLoss nn.LogSoft pytorch loss function

nn.NNLoss(x,y_target) = -x[y_target]

x=[1,2,3] y_target=[2] 则loss = x[2]=3

nll = nn.NLLLoss()
x = torch.randn(3,3)
y_target = torch.tensor([1,2,0])
cal_my = (x[0][1]+x[1][2] +x[2][0])/3
loss = nll(x,y_target)
print(x)
print(y_target)
print(loss)
print(cal_my)

x为:
tensor([[ 1.5827,  0.2479, -1.3481],
        [ 1.6454, -1.1276,  0.3725],
        [-0.9897, -1.4498,  2.3366]])

y_target为:
tensor([1, 2, 0])

loss为:
tensor(0.1231)

cal_my为:
tensor(0.1231)

 

function = nn.LogSoftmax (dim=1)

function (x) represents x let through Softmax, then through torch.log 

First, talk about the parameters of Softmax, calling Softmax function when you need to specify which dimension carried out in accordance with softmax, for example as follows:

x = torch.randn(3,3)
softmax_dim0 = nn.Softmax(dim=0)
softmax_dim1 = nn.Softmax(dim=1)

s_0 = softmax_dim0(x)
s_1 = softmax_dim1(x)
print(x)
print('s_0',s_0)
print('s_1',s_1)

输出结果为：
tensor([[ 0.3920,  0.6731,  1.0011],
        [-0.0462,  1.1450, -0.5482],
        [-0.2710,  1.1763, -1.1713]])
s_0 tensor([[0.4628, 0.2349, 0.7540],
        [0.2986, 0.3766, 0.1601],
        [0.2385, 0.3885, 0.0859]])
s_1 tensor([[0.2402, 0.3182, 0.4417],
        [0.2042, 0.6721, 0.1236],
        [0.1767, 0.7514, 0.0718]])

When the time set dim = 0, 0 in accordance with the first dimension for softmax 0.4628 + 0.2986 + 0.2385 = 1

When set to dim = 1 when performed 0.2402+ 0.3182+ 0.4417 = 1 softmax according to a first dimension

Verify nn.LogSoftmax following composition (dim = 1) and the log of softmax

 

softmax = nn.Softmax(dim=1)
logsoftmax = nn.LogSoftmax(dim=1)

x = torch.randn(3,3)
softmax_value = softmax(x)
cal_logsoftmax_value = torch.log(softmax_value)
logsoftmax_value = logsoftmax(x)

print('x',x)
print('softmax_value',softmax_value)
print('cal_logsoftmax_value',cal_logsoftmax_value)
print('logsoftmax_value',logsoftmax_value)

输出结果为：
x tensor([[-0.8330, -0.1150,  0.3054],
        [-0.2180, -1.1248,  0.4393],
        [-0.4476,  0.7618,  0.8175]])
softmax_value tensor([[0.1620, 0.3322, 0.5058],
        [0.3000, 0.1211, 0.5789],
        [0.1267, 0.4245, 0.4488]])
cal_logsoftmax_value tensor([[-1.8200, -1.1021, -0.6816],
        [-1.2040, -2.1108, -0.5467],
        [-2.0662, -0.8568, -0.8011]])
logsoftmax_value tensor([[-1.8200, -1.1021, -0.6816],
        [-1.2040, -2.1108, -0.5467],
        [-2.0662, -0.8568, -0.8011]])

 

function = nn.CrossEntroyLoss()

function (x) x is let go through after logsoftmax nlloss, verified as follows:

import torch
import torch.nn as nn

cel = nn.CrossEntropyLoss()
nll = nn.NLLLoss()
logsoftmax = nn.LogSoftmax(dim=1)

x = torch.randn(3,3)
y_target = torch.tensor([1,2,0])

logsoftmax_value = logsoftmax(x)
nll_value = nll(logsoftmax_value,y_target)
cel_value = cel(x,y_target)

print('x',x)
print('y_target',y_target)
print('logsoftmax_value',logsoftmax_value)
print('nll_value',nll_value)
print('cel_value',cel_value)


输出结果如下：
x tensor([[ 1.0021,  0.4425, -0.3256],
        [ 0.3272,  0.5255, -0.2736],
        [ 0.5227, -0.6293,  1.4442]])
y_target tensor([1, 2, 0])
logsoftmax_value tensor([[-0.6079, -1.1675, -1.9355],
        [-1.0180, -0.8197, -1.6188],
        [-1.3426, -2.4947, -0.4211]])
nll_value tensor(1.3763)
cel_value tensor(1.3763)