[MOT Study Notes] JDE Loss Function Detailed Explanation

I just wrote a paper recently and sorted it into the JDE algorithm. The loss function part of the original JDE paper is a bit vague.

(1) Loss function

Unlike YOLO v3, JDE uses a dual-threshold segmentation method to judge whether the target is foreground or background. That is, if the IoU of the target and a truth box is greater than 0.5, it is considered a match; if the IoU is less than 0.4, it is considered a mismatch. After experiments,
it is believed that this method can suppress false alarms (FP). For foreground and background classification loss ${\mathcal{L}}_{}$Using cross-entropy loss, the regression loss L β \mathcal{L}_{\beta} for the bounding box$L_{}$Smooth L1 loss is used,
as shown in formulas (4-1) and (4-2).
$$L_{}(x,y)=\frac{1}{N}\sum _{n=1}^N[-\sum _{c=1}^C\log \frac{e^{x_{n,c}}}{{\sum }_{i=1}^C{e}^{x_{n,i}}}]y_{n,c}$$
$${\mathcal{L}}_{}(x,y)=\frac{1}{N}\sum _{n=1}^N[\frac{1}{2}(x_n-y_n{)}^{2}I(|x_n-y_n|<1)+(|x_n-y_n|-0.5)\mathbb{I}(|x_n-y_n|\ge 1)]$$
where$x$ represents the prediction result,$y$ represents the truth value,$N$ represents the batch size,$e$ is the natural logarithm. xn in formula (4-1)$x_{n,c}$Indicates the predicted $x_n$belongs to category $The\; probability\; of\; c$ ,yn , c ∈ { 0 , 1 } y_{n,c}\in\{0,1\}yn,c​∈{ 0,1 } means labelyyDoes$y$$c$ . I ( ⋅ ) \mathbb{I}(·)in formula (4-2)$\mathbb{I}(\cdot )$ is an indicator function.

For the appearance feature learning task, the desired effect is that the distance metric should be large enough for different objects. JDE treats this problem as a classification problem. Suppose the target number of different instances in the entire video sequence is $nID$ , then the algorithm shouldn ID nID
on the target by embedding vectorClassification of$n$$I$$D\; categories.$

Suppose an anchor instance in a batch of samples is $f^T$ , the positive sample (that is, the true value category) is$f^{+}$ , which with the anchor$f^T$ is related; negative samples (that is, other categories) are$f^{-}$ . When calculating the loss, pay attention to all negative sample classifications. Take$f^T{f}^{+}$ indicates the probability that the anchor instance is
considered as a positive sample,$f^T{f}_j^{-}$Indicates that the anchor is considered to be the $The\; probability\; of\; j$ categories, the loss is calculated in a form similar to the cross-entropy function:
$${\mathcal{L}}_{}(x,y)=\frac{1}{N}\sum _{i=1}^N[-\log \frac{e^{f_i^T{f}_i^{+}}}{e^{f_i^T{f}_i^{+}}+{\sum }_j{e}^{f_i^T{f}_{i,j}^{-}}}]$$
where subscript$i$ display$i$ samples.

(2) Loss balance

JDE learns three tasks simultaneously: classification, bounding box regression, and appearance feature learning. Therefore, how to balance the three tasks is a very important issue. Most of the other algorithms are weighted sums of the loss functions of each part, but JDE adopts the method of automatically adjusting the importance of multi-tasks to select the weight of each part of the loss. Specifically, refer to the concept of task-independent uncertainty proposed in [39] to learn the weights of each part of the loss as network parameters.
Therefore, the total loss function is shown in formula (4-4):
$${\mathcal{L}}_{total}=\sum _{i=1}^M\sum _{j=a,b,c}\frac{1}{2}(\frac{1}{e^{s_j^i}}{L}_j^i+s_j^i)$$
in that$s_j^i$is a task-independent uncertainty and is a learnable parameter. $M$ is the number of tasks, since there are three tasks of classification, bounding box regression and appearance feature learning, so$M=3$.