Target detection performance evaluation index

1. Common model evaluation terms

Now assume that our classification target has only two categories, counted as positive examples (positive) and negative examples (negative):

True positives(TP)

The number of correctly classified positive examples, that is, the number of instances (number of samples) that are actually positive examples and classified as positive examples by the classifier;

False positives(FP)

The number of instances that are incorrectly classified as positive instances, that is, the number of instances that are actually negative instances but are classified as positive instances by the classifier;

False negatives(FN)

The number of instances that are incorrectly classified as negative examples, that is, the number of instances that are actually positive examples but are classified as negative examples by the classifier;

True negatives(TN)

The number of instances that are correctly classified as negative examples, that is, the number of instances that are actually negative examples and are classified as negative examples by the classifier.

P-R curve

• P：

  It represents precision, which is the accuracy rate. The calculation formula is:
  the actual number of positive samples in the predicted sample/the number of all positive samples, that is, precision=TP/(TP+FP)

• R: stands for recall, which is the recall rate. The calculation formula is:

  The actual number of positive samples in the predicted sample/the number of predicted samples is Recall=TP/(TP+FN)=TP/P.
  Generally speaking, the relationship between precision and recall is a fish-and-cake relationship. The higher the recall rate, the better the accuracy rate. Low

PR curve

That is, a two-dimensional curve with precision and recall as the vertical and horizontal axis coordinates. The overall trend is drawn by selecting the corresponding precision and recall rates at different thresholds. The higher the precision, the lower the recall. When the recall reaches 1, it corresponds to the positive sample with the lowest probability score. At this time, the number of positive samples is divided by all those greater than or equal to the threshold. The number of samples is the lowest precision value.
In addition, the area enclosed by the PR curve is the AP value. Generally speaking, a better classifier has a higher AP value. In target detection, each class can draw a PR curve based on recall and precision. AP is the area under the curve, and mAP is the average AP of all classes.

AP

That is Average Precision, that is, average accuracy

mAP

That is, Mean Average Precision is the average AP value, which is the average AP value of multiple verification set individuals, and is used as an indicator to measure detection accuracy in object detection.

IOU (intersection and union ratio)

Intersection-over-Union is a concept used in target detection and is a standard for measuring the accuracy of detecting corresponding objects in a specific data set.
IOU represents the overlap rate or degree of overlap between the generated candidate box (candidate bound) and the original marked box (ground truth bound), that is, the ratio of their intersection and union. The higher the correlation, the value. The ideal situation is complete overlap, that is, the ratio is 1. Calculated as follows:

$$\mathrm{IoU}=\frac{\mathrm{area}\left(C\right)\cap \mathrm{area}(G)}{\mathrm{area}(C)\cup \mathrm{area}(G)}$$

NMS (non-maximum suppression)

That is, non-maximum suppression means non-maximum suppression. As the name suggests, it suppresses elements that are not maximum values ​​and searches for local maximum values.
In object detection, NMS is widely used. Its purpose is to remove redundant frames and find the best object detection position.

Detection speed

• Prequel takes time
• Frames per second FPS (Frame Per Second)
• Floating point operations (FLOPS)

In addition to detection accuracy, another important performance indicator of the target detection algorithm is speed. Only with fast speed can real-time detection be achieved, which is extremely important for some application scenarios. A common metric for evaluating speed is Frame Per Second (FPS), which is the number of images that can be processed per second. Of course, to compare FPS, you need to do it on the same hardware. In addition, the time required to process an image can also be used to evaluate the detection speed. The shorter the time, the faster the speed.