Method

A Multi-Modal Fusion-Based 3D Multi-Object Tracking Framework with Joint Detection [YONTD-MOTv2]
https://github.com/wangxiyang2022/YONTD-MOT

Submitted on 15 Jul. 2024 16:41 by
Xiyang Wang (Chongqing University (CQU SLAMMOT Team))

Running time:0.1 s
Environment:GPU @ >3.5 Ghz (Python)

Method Description:
Firstly, a new multi-object tracking framework is
proposed in this paper based on multi-modal
fusion. By integrating object detection and multi-
object tracking into the same model, this
framework avoids the complex data association
process in the classical TBD paradigm, and
requires no additional training. Secondly,
confidence of historical trajectory regression is
explored, possible states of a trajectory in the
current frame (weak object or strong object) are
analyzed and a confidence fusion module is
designed to guide non-maximum suppression of
trajectory and detection for ordered association.
Finally, extensive experiments are conducted on
the KITTI and Waymo datasets. The results show
that the proposed method can achieve robust
tracking by using only two modal detectors and it
is more accurate than many of the latest TBD
paradigm-based multi-modal tracking methods. The
source codes of the proposed method are available
at https://github.com/wangxiyang2022/YONTD-
Parameters:
TBD
Latex Bibtex:
@article{wang2024multi,
title={A Multi-Modal Fusion-Based 3D Multi-
Object Tracking Framework with Joint Detection},
author={Wang, Xiyang and Fu, Chunyun and He,
Jiawei and Huang, Mingguang and Meng, Ting and
Zhang, Siyu and Zhou, Hangning and Xu, Ziyao and
Zhang, Chi},
journal={IEEE Robotics and Automation Letters},
year={2024},
publisher={IEEE}
}

Detailed Results

From all 29 test sequences, our benchmark computes the HOTA tracking metrics (HOTA, DetA, AssA, DetRe, DetPr, AssRe, AssPr, LocA) [1] as well as the CLEARMOT, MT/PT/ML, identity switches, and fragmentation [2,3] metrics. The tables below show all of these metrics.


Benchmark HOTA DetA AssA DetRe DetPr AssRe AssPr LocA
CAR 79.52 % 75.83 % 84.01 % 82.31 % 83.69 % 87.16 % 90.70 % 87.65 %

Benchmark TP FP FN
CAR 32075 2317 1751

Benchmark MOTA MOTP MODA IDSW sMOTA
CAR 88.06 % 86.24 % 88.17 % 37 75.23 %

Benchmark MT rate PT rate ML rate FRAG
CAR 80.15 % 17.23 % 2.62 % 363

Benchmark # Dets # Tracks
CAR 33826 849

This table as LaTeX


This figure as: png pdf

[1] J. Luiten, A. Os̆ep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, B. Leibe: HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. IJCV 2020.
[2] K. Bernardin, R. Stiefelhagen: Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. JIVP 2008.
[3] Y. Li, C. Huang, R. Nevatia: Learning to associate: HybridBoosted multi-target tracker for crowded scene. CVPR 2009.


eXTReMe Tracker