[pytorch] nn.SmoothL1Loss function use

---

1、Smooth L1 Loss

This method was proposed by the Microsoft rgb master, and the Fast RCNN paper proposed this method

1) Assuming that x is the numerical difference between the predicted frame and the real frame, the commonly used definitions of L1 loss, L2 Loss and smooth L1 loss are:

2) The derivatives of the above three loss functions to x are:

From the derivative of the loss function with respect to x, we know:

• $L1$ loss function forxxThe derivative of$x$$When\; x$ is small, if the learning rate remains unchanged, the loss function will fluctuate around a stable value, making it difficult to converge to a higher accuracy.
• $The\; derivative\; of\; the\; L2$ loss function to x is very large when the value of x is large, and it is unstable in the early stage of training.
• $smoot{h}_L$Perfect to avoid $L1$ and$The\; downside\; of\; L2$ loss$.$

3) The loss loss in the actual target detection frame regression task is: $L_{loc(t^u,v)}={\sum }_{i\in (x,y,w,h)}smoot{h}_L(t_i^u-v_i)$

in:

• $v=(v_x,v_y,v_w,v_h)$ : Indicates the frame coordinates of GT,
• $t^u=(t_x^u,t_y^u,t_x^w,t_h^u)$ : Indicates the predicted box coordinates, that is, the loss of the four points is calculated separately, and then added together as the Bounding Box Regression Loss.

---

2. Use of nn.SmoothL1Loss function

torch.nn.SmoothL1Loss(size_average=None, reduce=None, reduction='mean', beta=1.0)

parameter:

• size_averageand reducehave been deprecated, the specific function can be replaced by reduction
• reduction: Specify the form of loss output, there are three options: none|mean|sum.
  • none: the loss does not do any processing, and directly outputs an array
  • mean: Calculate the average of the obtained losses and output, and output a number
  • sum: Sum the obtained losses and output, and output a number
• beta: Specify the loss in $L1\sim L_2$The threshold to vary between, defaults to 1.0

import torch.nn as nn
import torch

loss1 = nn.SmoothL1Loss(reduction='none')
loss2 = nn.SmoothL1Loss(reduction='mean')

y = torch.randn(3)
y_pred = torch.randn(3)
loss_value1 = loss1(y, y_pred)
loss_value2 = loss2(y, y_pred)

print(y)   # tensor([ 1.6938, -0.3617, -1.2738])
print(y_pred)   # tensor([ 0.3932,  0.8715, -0.2410])
print(loss_value1)   # tensor([0.8007, 0.7332, 0.5328])
print(loss_value2)   # tensor(0.6889)