Calculation of AP and MAP in target detection

$$MAP\left(flataveragequasicorrectrate\right)$$

First recommend ## https://github.com/rafaelpadilla/Object-Detection-Metrics

This, what is said here is actually quite clear.

To really understand the calculation of mAP, you have to understand these concepts

One is the concept of confidence and IOU threshold

Through the confidence and IOU threshold, you can determine TP and FP

2. Understand the concept of FN

In testing, it is necessary to understand the concept of GroundTruth total

Three Precision and Recall

Knowing FN, TP, FP can know Precision and Recall
precisoin = TP / (TP+FP)
recall = TP / (TP+FN)

Four understand what is called 11-point interpolation display method

---

In fact, there is a question worth thinking about, why recall can be used as the abscissa, does recall only increase but not decrease?

Sorry, I recall, I really only go up and not down, I just do whatever I want

Because the detection frame will be sorted in descending order according to the confidence level, the recall will only increase but not decrease, which can be used as the abscissa

---

If you understand and calculate the Precision and Recall of each prediction box, you can draw准确度x召回率曲线

The 11-point interpolation display method is to divide the [0, 1] area into 10 points evenly.

From the above figure, we can see that all the detection frames to the last drawn curve are in the recall range [0-0.5]

Next, using the 11-point interpolation display method, first determine that the abscissa distribution of 11 points is [0, 0.1, 0.2, 0.3,..., 1.0]

Then according to each horizontal axis, and the above picture 准确度x召回率曲线, determine the ordinate of the red point in the picture below, and then make the picture below

AP

---

Establish the concept of Precision and Recall

Precision

Precision is the proportion of all your predictions that are true in the samples that are truly correct
(that is, suppose you predict 100 samples are true, but there may only be 80 of them that are true. There are 20 false, but these 20 you predicted to be true, so your Precision is 80 / (80 + 20) = 0.8)

Note: You predicted 100 samples. These 100 samples can be selected from the original 2000 samples

Recall

Recall refers to the proportion of the number of samples that are truly correct among the samples that you predict is true, and the proportion of the number of samples that you are truly correct and the sum of your false predictions.

What is the sum of the number of samples whose predictions are really correct and your false predictions?

Suppose you predict 200 samples. Of these 200 samples, you predict 100 as true and 100 as false. Among the 100 samples that you predict as true, 80 are true and the prediction is true. There are 20 fake ones. Among the 100 predictions that are false, 30 of them are true, but your predictions are false. This is the number of false predictions.

ok! Here comes the point! ! !

One predicts 200 samples, the prediction is true or false, and each is divided into 100 samples!
Among the 100 predictions that are true, 80 are really true, and
among the 100 predictions that are false, the number that is truly true, that is, the number of false predictions is 30.

So what is the sum of the number of samples whose predictions are really correct and your false predictions?
That is 80+30, which is the number of samples that are truly true in these 200!

Why is it said that Precision and Recall are in conflict

Because all the predictions are false, take the example above, it is 100, 100 = FP+FN, if the total number of all predictions is false, the FP is more, the FN is less; the FN is more, the FP is less

Detection index: calculation of AP value

Second, establish the concept of confidence and IOU

• Confidence: the probability that the framed area is the object we need
• IOU: I understand everything

---

• True Positive (TP): IOU>= threshold detection box
• False Positive (FP): IOU<threshold detection box
• False Negative (FN): GT that has not been detected
• True Negative (TN): ignore

The calculation of the detection index needs to be based on the confidence level. The IOU threshold here is 0.5 by default. As for the IOU value in the cat picture, it is assumed to be 0.76, 0, and 0.91 respectively.

As shown by Xiao Miaomiao above: the confidence distribution is [0.7, 0.8, 0.9]

---

When the confidence is 0.85

When the confidence level is 0.85, only cat3 is detected, so the detection box is left with one iou=0.91 in cat3

• True Positive (TP): IOU>= threshold detection box
• False Positive (FP): IOU<threshold detection box
• False Negative (FN): GT that has not been detected

Here iou threshold = 0.5

TP = 1
FP = 0 Missed inspection
:
FN = 2 (green box in cat1 and cat2)

---

Precision = TP/(TP+FP) =1/1
Recall =TP/(TP+FN) =1/3

---

When the confidence is 0.75

When the confidence level is 0.75, cat3 and cat2 are detected, so the detection box is left with iou=0.91 and iou=0 in cat3 and cat2.

• True Positive (TP): IOU>= threshold detection box
• False Positive (FP): IOU<threshold detection box
• False Negative (FN): GT that has not been detected

Here iou threshold = 0.5

TP = 1
FP = 1

Missed inspection : FN = 2 (green box in cat1 and cat2)

---

Precision = TP/(TP+FP) =1/2
Recall =TP/(TP+FN) =1/3

---

When the confidence is 0.5

When the confidence level is 0.5, all the cat images are considered to be detected, iou=0.91, iou=0, iou=0.76 in cat3, cat2, and cat1

• True Positive (TP): IOU>= threshold detection box
• False Positive (FP): IOU<threshold detection box
• False Negative (FN): GT that has not been detected

Here iou threshold = 0.5

TP = 2
FP = 1

Missed inspection : FN = 1 (green box in cat2)

---

Precision = TP/(TP+FP) =2/3
Recall =TP/(TP+FN) =2/3

---

In fact, it can be seen from the above that every change in the confidence or IOU threshold will cause the recalculation of TP and FP, that is, the recalculation of recall and precision.

---

Calculate AP through recall and precision

Calculate the change of precision value under different recall values

Attention, attention, what is each recall and precision?

How is the recall value of each point below determined?
I don’t know, hahahaha!

---

https://github.com/rafaelpadilla/Object-Detection-Metrics

---

In order to calculate precision and recall, you need
True Positives: the prediction is true, but it is true
False Negatives: the prediction is false, but it is true, and it is missed

False Positives: The prediction is true, but it is actually false.
True Negatives: The prediction is false, but it is actually false.

---

TP + FN = all true

TP/(TP+FN) = Among all the truths, the proportion of predictions that are true

---

TP+FP = all predictions are true

TP/(TP+FP)
among all predictions that are true, the true ratio

---

If IoU> 0.5, it is considered as True Positive, otherwise it is considered as False Positive

---

For each picture, the ground truth data will give the actual number of objects in each category in the picture, calculate the IoU value of each Positive prediction box and ground truth, and take the largest IoU value, which is considered to be the largest IoU detected by the prediction box Ground truth. Then according to the IoU threshold, we can calculate the number of correct detection values ​​(True Positives, TP) and the number of false positives (False Positives, FP) for each category in a picture.

Attention 类别concept

Based on this, each 类别precision can be calculated

---

The number of correct predicted values ​​(True Positives), and the number of objects that were missed (False Negatives, FN)
can be easily calculated. Recall can be calculated (the denominator can be the total number of ground truth)

---

In object detection, the definition of mAP first appeared in the PASCAL Visual Objects Classes (VOC) competition

---

I feel like a bunch of rubbish talk below, I can’t read it

$$The\; greencolorof\; thetruesolidblock,a\; redcolorofapre-measuredblock$$

There are 7 images, of which 15 ground truth objects are represented by green bounding boxes, and 24 detected objects are represented by red bounding boxes. Each detected object is identified by letters (A, B,..., Y) and has confidence

Then sort BBOX by confidence

Then calculate the corresponding precision and recall values ​​for each BBOX

Examples:

• 对BBOXR，precision = 1/1 =1，recall = 1/15=0.0666
• For BBOXY, precision = 1/2=0.5, recall = 1/15=0.0666
• 对BBOXJ，precision = 2/3 =-0.666s，recall =2/15=0.1333

---

After 2012, use area to approximate AP value

---

Definition of TP, FP, FN, TN in target detection

• True Positive (TP): IOU>= threshold detection box
• False Positive (FP): IOU<threshold detection box
• False Negative (FN): GT that has not been detected
• True Negative (TN): ignore

---