Focal Loss paper study notes

原论文：《T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Doll´ar. Focal loss for dense object detection[C]. In ICCV, 2017.》

• A special target detection network RetinaNet is specially designed for experimental verification.
• After the paper experiments RetinaNet loss of function parameters over one pair optimum value : $c=2$、$a=0.25$。

Purpose : To solve the imbalance problem of positive and negative samples in the target detection model .
Method : Through the loss function, the role of the correctly classified samples in the weight update is suppressed. The higher the score of a sample that is correctly classified, the lower its effect .

For example: a positive sample with a score of 0.99 has less effect than a positive sample with a score of 0.70; a negative sample with a score of 0.01 has less effect than a negative sample with a score of 0.40.

1. Focal Loss loss function

First introduced cross-entropy loss function (Cross Entropy) and balanced cross-entropy loss function (Balanced Cross Entropy), these two functions of the proposed Focal Loss loss function was an eye opener.

记$y$ is the positive and negative category, 1 is positive and 0 is negative;$p$ is the predicted value of the model for the positive category,$1-p$ is the predicted value for the negative category.

• Cross entropy loss function
  $$CE(p,and)=\{\begin{array}{ll}& -log(p)ify=1\\ & -log(1-p)ify=0\end{array}$$
  记
  $$p_t=\{\begin{array}{ll}& pify=1\\ & 1-pify=0\end{array}$$
  则
  $$CE(p,and)=CE(p_t)=-log(p_t)$$
• Balanced cross entropy loss function
  $$CE\left(p_t\right)=-{\alpha }_{t}log(p_t)$$
  Where$a\in [0,1]$，且
  $$a_t=\{\begin{array}{ll}& \alpha ify=1\\ & 1-\alpha ify=0\end{array}$$

1.1 Focal Loss loss function

$$FL(p_t)=-(1-p_t{)}^{\gamma }log(p_t)$$
$c\ge 0$ is a hyperparameter, which is called Tunable Focusing Parameter. The following is the loss function curve under the control of different values. In the experiment of the paper,$c=2$ has the best effect.

1.2 Based on $\alpha$ 's Focal Loss loss function

This is the actual loss function used.
$$FL(p_t)=-{\alpha }_t(1-p_t{)}^{\gamma }log(p_t)$$

2. RetinaNet Network

Structure diagram:

The network will not be described in detail.

Finally, the experimental data: