Use the MOT evaluation tool TrackEval to evaluate your own data set

TrackEval tutorial

You can read what this blogger wrote, very detailed https://zhuanlan.zhihu.com/p/391396206

Make your own MOT dataset

software recommendation

Here I recommend using Darklabel . If you can’t get in, I will upload it to Baidu Cloud for everyone to download and use. Link:

Link: https://pan.baidu.com/s/1K7vWRWr6jzi6xFSfAxwhVA
Extraction code: phu

1. Open the yaml file and write in the categories you want to track. My categories here are the following categories

my_classes: ["uav", "car", "person"]

2. Relevant explanations , but now the version shortcut key is shift+mouse operation.
After making relevant data sets, track the completion and save the results. Use TrackEval for evaluation.
The format of our dataset is as follows:

1,1,592,444,482,284,1,-1,-1,-1

Name it gt.txt

file path

#gt信息
#如果多个的话，并列存放
data/
    gt/
        mot_challenge/
            MyDataset/
                seq-01/   # 视频名
                    gt/
                        gt.txt          # <---- ground truth
                    seqinfo.ini         # 放你的视频的信息
    trackers/   # 你自己代码运行出来的结果
        mot_challenge/
            MyDataset/
                data/
                    seq-01.txt          # <---- model result 视频结果.txt

Seqinfo.ini Information

[Sequence]
name=ai_city
imDir=img1
frameRate=30
seqLength=1996
imWidth=1920
imHeight=1080
imExt=.jpg

run

run scripts/run_mot_challenge.py

-GT_FOLDER # gt路径
--BENCHMARK ai_city   # 视频名
--DO_PREPROC False 
--METRICS HOTA # 选择测评指标 'HOTA', 'CLEAR', 'Identity'

Here is the result I ran out myself, because I used both gt.txt

CLEAR: data-pedestrian             MOTA      MOTP      MODA      CLR_Re    CLR_Pr    MTR       PTR       MLR       sMOTA     CLR_TP    CLR_FN    CLR_FP    IDSW      MT        PT        ML        Frag      
more_2_0                           100       100       100       100       100       100       0         0         100       1056      0         0         0         4         0         0         0         
more_2_1                           100       100       100       100       100       100       0         0         100       1065      0         0         0         5         0         0         0         
more_2_2                           100       100       100       100       100       100       0         0         100       476       0         0         0         2         0         0         0         
more_2_3                           100       100       100       100       100       100       0         0         100       971       0         0         0         4         0         0         0         
COMBINED                           100       100       100       100       100       100       0         0         100       3568      0         0         0         15        0         0         0         

Count: data-pedestrian             Dets      GT_Dets   IDs       GT_IDs    
more_2_0                           1056      1056      4         4         
more_2_1                           1065      1065      5         5         
more_2_2                           476       476       2         2         
more_2_3                           971       971       4         4         
COMBINED                           3568      3568      15        15   

Subsequent bugs found

During the evaluation again, the following problem appeared, and some people also encountered it in the comment area. The error reported in the problem was because the category was wrong. Therefore, the category in the gt file needs to be modified. After we use Darklabel to mark, the format of the gt file yes:

1,1,592,444,482,284,-1,-1,-1,-1

We need to modify it to

1,1,592,444,482,284,-1,1,-1,-1

Finally my tests pass.

trackeval.utils.TrackEvalException: Attempting to evaluate using invalid gt classes. This warning only triggers if preprocessing is performed, e.g. not for MOT15 or where prepropressing is explicitly disabled. Please either check your gt data, or disable preprocessing. The following invalid classes were found in timestep 1: -1