Could not evaluate the given results when i evaluate

[wzgliang]

when i evaluate the trained result, i use python scripts/evaluate.py , and at begin, the program goes well, then, the program Could not evaluate the given results. As following:

(mot_neural_solver) root@autodl-container-377e11abac-03f90540:~/mot_neural_solver# python scripts/evaluate.py
WARNING - evaluate - No observers have been added to this run
INFO - evaluate - Running command 'main'
INFO - evaluate - Started
Successfully loaded pretrained weights from "/root/mot_neural_solver/output/trained_models/reid/resnet50_market_cuhk_duke.tar-232"
** The following layers are discarded due to unmatched keys or layer size: ['classifier.weight', 'classifier.bias']
Loading processed dets for sequence TUD-Crossing from /root/mot_neural_solver/data/2DMOT2015/test/TUD-Crossing/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence PETS09-S2L2 from /root/mot_neural_solver/data/2DMOT2015/test/PETS09-S2L2/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence ETH-Jelmoli from /root/mot_neural_solver/data/2DMOT2015/test/ETH-Jelmoli/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence ETH-Linthescher from /root/mot_neural_solver/data/2DMOT2015/test/ETH-Linthescher/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence ETH-Crossing from /root/mot_neural_solver/data/2DMOT2015/test/ETH-Crossing/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence AVG-TownCentre from /root/mot_neural_solver/data/2DMOT2015/test/AVG-TownCentre/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence ADL-Rundle-1 from /root/mot_neural_solver/data/2DMOT2015/test/ADL-Rundle-1/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence ADL-Rundle-3 from /root/mot_neural_solver/data/2DMOT2015/test/ADL-Rundle-3/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence KITTI-16 from /root/mot_neural_solver/data/2DMOT2015/test/KITTI-16/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence KITTI-19 from /root/mot_neural_solver/data/2DMOT2015/test/KITTI-19/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence Venice-1 from /root/mot_neural_solver/data/2DMOT2015/test/Venice-1/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-01-DPM from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-01-DPM/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-01-FRCNN from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-01-FRCNN/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-01-SDP from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-01-SDP/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-03-DPM from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-03-DPM/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-03-FRCNN from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-03-FRCNN/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-03-SDP from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-03-SDP/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-06-DPM from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-06-DPM/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-06-FRCNN from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-06-FRCNN/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-06-SDP from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-06-SDP/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-07-DPM from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-07-DPM/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-07-FRCNN from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-07-FRCNN/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-07-SDP from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-07-SDP/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-08-DPM from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-08-DPM/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-08-FRCNN from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-08-FRCNN/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-08-SDP from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-08-SDP/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-12-DPM from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-12-DPM/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-12-FRCNN from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-12-FRCNN/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-12-SDP from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-12-SDP/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-14-DPM from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-14-DPM/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-14-FRCNN from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-14-FRCNN/processed_data/det/tracktor_prepr_det.pkl
Loading processed dets for sequence MOT17-14-SDP from /root/mot_neural_solver/data/MOT17Labels/test/MOT17-14-SDP/processed_data/det/tracktor_prepr_det.pkl
Tracking TUD-Crossing
Tracking sequence  TUD-Crossing
Done!

Tracking PETS09-S2L2
Tracking sequence  PETS09-S2L2
Done!

Tracking ETH-Jelmoli
Tracking sequence  ETH-Jelmoli
Done!

Tracking ETH-Linthescher
Tracking sequence  ETH-Linthescher
Done!

Tracking ETH-Crossing
Tracking sequence  ETH-Crossing
Done!

Tracking AVG-TownCentre
Tracking sequence  AVG-TownCentre
Done!

Tracking ADL-Rundle-1
Tracking sequence  ADL-Rundle-1
Done!

Tracking ADL-Rundle-3
Tracking sequence  ADL-Rundle-3
Done!

Tracking KITTI-16
Tracking sequence  KITTI-16
Done!

Tracking KITTI-19
Tracking sequence  KITTI-19
Done!

Tracking Venice-1
Tracking sequence  Venice-1
Done!

Tracking MOT17-01-DPM
Tracking sequence  MOT17-01-DPM
Done!

Tracking MOT17-01-FRCNN
Tracking sequence  MOT17-01-FRCNN
Done!

Tracking MOT17-01-SDP
Tracking sequence  MOT17-01-SDP
Done!

Tracking MOT17-03-DPM
Tracking sequence  MOT17-03-DPM
Done!

Tracking MOT17-03-FRCNN
Tracking sequence  MOT17-03-FRCNN
Done!

Tracking MOT17-03-SDP
Tracking sequence  MOT17-03-SDP
Done!

Tracking MOT17-06-DPM
Tracking sequence  MOT17-06-DPM
Done!

Tracking MOT17-06-FRCNN
Tracking sequence  MOT17-06-FRCNN
Done!

Tracking MOT17-06-SDP
Tracking sequence  MOT17-06-SDP
Done!

Tracking MOT17-07-DPM
Tracking sequence  MOT17-07-DPM
Done!

Tracking MOT17-07-FRCNN
Tracking sequence  MOT17-07-FRCNN
Done!

Tracking MOT17-07-SDP
Tracking sequence  MOT17-07-SDP
Done!

Tracking MOT17-08-DPM
Tracking sequence  MOT17-08-DPM
Done!

Tracking MOT17-08-FRCNN
Tracking sequence  MOT17-08-FRCNN
Done!

Tracking MOT17-08-SDP
Tracking sequence  MOT17-08-SDP
Done!

Tracking MOT17-12-DPM
Tracking sequence  MOT17-12-DPM
Done!

Tracking MOT17-12-FRCNN
Tracking sequence  MOT17-12-FRCNN
Done!

Tracking MOT17-12-SDP
Tracking sequence  MOT17-12-SDP
Done!

Tracking MOT17-14-DPM
Tracking sequence  MOT17-14-DPM
Done!

Tracking MOT17-14-FRCNN
Tracking sequence  MOT17-14-FRCNN
Done!

Tracking MOT17-14-SDP
Tracking sequence  MOT17-14-SDP
Done!

Could not evaluate the given results
INFO - evaluate - Completed after 0:05:46
(mot_neural_solver) root@autodl-container-377e11abac-03f90540:~/mot_neural_solver#

thanks a lot and look forward ur reply! It's very important to me!

[aksloms]

Hey, I have the same issue. Looks like grund truth sequences do not align with test sequences

[guillembraso]

Hey, you are running the evaluation with the MOT15/MOT17 test sequences (check out this config entry). There is no ground truth data for those sequences (you have to submit to the online test server). That is why they cannot be evaluated