introduction

There are three main categories of semantic segmentation evaluation indicators, namely execution time, memory usage and accuracy. This article mainly introduces several indicators and python code implementation:

F-score/DICE

F-score, also known as the Dice coefficient or QS (similar quotient), is a measure of the similarity between two sets. For semantic segmentation tasks, it is used to evaluate the segmentation results predicted by the network and the human-labeled results. of similarity. F-score is equal to 2 times the product of precision and recall divided by the sum of precision and recall.
$\frac{2 \times P \times R}{P+ R}$

# 计算F1-score(F1)
def f1_accuracy(pred, label):
    # pred和label都是二维数组，表示预测和真实的像素类别
    # 返回一个0~1之间的浮点数，表示像素准确率
    # 计算混淆矩阵
    cm = confusion_matrix(y_true.flatten(), y_pred.flatten())
    # 计算F1分数
    return f1_score(y_true.flatten(), y_pred.flatten())

PA：

Pixel accuracy is a measure of the proportion of correctly classified pixels to the total number of pixels in the segmentation result. PA is equal to the sum of diagonal elements in the confusion matrix divided by the sum of all elements in the confusion matrix.

# 计算像素准确率(PA)
def pixel_accuracy(pred, label):
    # pred和label都是二维数组，表示预测和真实的像素类别
    # 返回一个0~1之间的浮点数，表示像素准确率
    assert pred.shape == label.shape
    return (pred == label).mean()

CPA:

Class pixel accuracy is a measure of the proportion of correctly classified pixels in each class to the total number of pixels in that class. CPA is equal to dividing the i-th row and i-column element in the confusion matrix by the sum of all elements in the i-th row in the confusion matrix.

# 计算类别像素准确率（CPA）
def class_pixel_accuracy(pred, label, num_classes):
    # pred和label都是二维数组，表示预测和真实的像素类别
    # num_classes是一个整数，表示类别数目
    # 返回一个长度为num_classes的一维数组，表示每个类别的像素准确率
    assert pred.shape == label.shape
    cpa = np.zeros(num_classes)
    for i in range(num_classes):
        cpa[i] = (pred[label == i] == i).mean()
    return cpa

MPA:

Category Average Pixel Accuracy, which is a measure of the average CPA of all categories. MPA is equal to the sum of all category CPAs divided by the number of categories.

# 计算类别平均像素准确率（MPA）
def mean_pixel_accuracy(pred, label, num_classes):
    # pred和label都是二维数组，表示预测和真实的像素类别
    # num_classes是一个整数，表示类别数目
    # 返回一个0~1之间的浮点数，表示类别平均像素准确率
    assert pred.shape == label.shape
    mpa = class_pixel_accuracy(pred, label, num_classes).mean()
    return mpa

IOU:

The intersection-union ratio is a measure of the degree of overlap between two sets. For semantic segmentation tasks, it is used to evaluate the degree of overlap between the segmentation results predicted by the network and the human-labeled results. IOU is equal to the intersection area of two sets divided by the union area of two sets.

# 计算交并比（IoU）
def intersection_over_union(pred, label, num_classes):
    # pred和label都是二维数组，表示预测和真实的像素类别
    # num_classes是一个整数，表示类别数目
    # 返回一个长度为num_classes的一维数组，表示每个类别的交并比
    assert pred.shape == label.shape
    iou = np.zeros(num_classes)
    for i in range(num_classes):
        intersection = ((pred == i) & (label == i)).sum()
        union = ((pred == i) | (label == i)).sum()
        if union > 0:
            iou[i] = intersection / union
        else:
            iou[i] = np.nan
            # 表示该类别不存在于图像中
    return iou

MEOW

The average IOU is a measure of the average of all categories of IOU. MIOU is equal to the sum of all category IOUs divided by the number of categories.

# 计算平均交并比（MIoU）
def mean_intersection_over_union(pred, label, num_classes):
    # pred和label都是二维数组，表示预测和真实的像素类别
    # num_classes是一个整数，表示类别数目
    # 返回一个0~1之间的浮点数，表示平均交并比
    assert pred.shape == label.shape
    miou = intersection_over_union(pred, label, num_classes).mean()
    return miou

Python implementation of semantic segmentation index system: F-score/DICE, PA, CPA, MPA, IOU, MIOU