Method

3D Multi-Object Tracking: A Baseline and New Evaluation Metrics [AB3DMOT+PointRCNN]
https://github.com/xinshuoweng/AB3DMOT

Submitted on 26 Feb. 2022 23:01 by
Xinshuo Weng (Carnegie Mellon University)

Running time:0.0047s
Environment:1 core @ 2.5 Ghz (python)

Method Description:
3D multi-object tracking (MOT) is an essential
component for many applications such as
autonomous driving and assistive robotics. Recent
work on 3D MOT focuses on developing accurate
systems giving less attention to practical
considerations such as computational cost and
system complexity. In contrast, this work
proposes a simple real-time 3D MOT system. Our
system first obtains 3D detections from a LiDAR
point cloud. Then, a straightforward combination
of a 3D Kalman filter and the Hungarian algorithm
is used for state estimation and data
association. Additionally, 3D MOT datasets such
as KITTI evaluate MOT methods in the 2D space and
standardized 3D MOT evaluation tools are missing
for a fair comparison of 3D MOT methods.
Therefore, we propose a new 3D MOT evaluation
tool along with three new metrics to
comprehensively evaluate 3D MOT methods. We show
that, although our system employs a combination
of classical MOT modules, we achieve state-of-
the-art 3D MOT performance on two 3D MOT
Parameters:
python=3.6
Latex Bibtex:
@article{Weng2020_AB3DMOT,
author = {Weng, Xinshuo and Wang, Jianren and Held,
David and Kitani, Kris},
journal = {IROS},
title = {{3D Multi-Object Tracking: A Baseline and
New Evaluation Metrics}},
year = {2020}
}

Detailed Results

From all 29 test sequences, our benchmark computes the HOTA tracking metrics (HOTA, DetA, AssA, DetRe, DetPr, AssRe, AssPr, LocA) [1] as well as the CLEARMOT, MT/PT/ML, identity switches, and fragmentation [2,3] metrics. The tables below show all of these metrics.


Benchmark HOTA DetA AssA DetRe DetPr AssRe AssPr LocA
CAR 69.99 % 71.13 % 69.33 % 75.66 % 84.40 % 72.31 % 89.02 % 86.85 %
PEDESTRIAN 37.81 % 32.37 % 44.33 % 34.91 % 59.35 % 48.44 % 62.83 % 71.31 %

Benchmark TP FP FN
CAR 29849 4543 979
PEDESTRIAN 11314 11836 2305

Benchmark MOTA MOTP MODA IDSW sMOTA
CAR 83.61 % 85.23 % 83.94 % 113 70.80 %
PEDESTRIAN 38.13 % 64.54 % 38.92 % 181 20.80 %

Benchmark MT rate PT rate ML rate FRAG
CAR 66.92 % 24.00 % 9.08 % 206
PEDESTRIAN 23.02 % 33.33 % 43.64 % 879

Benchmark # Dets # Tracks
CAR 30828 785
PEDESTRIAN 13619 313

This table as LaTeX


This figure as: png pdf

This figure as: png pdf

[1] J. Luiten, A. Os̆ep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, B. Leibe: HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. IJCV 2020.
[2] K. Bernardin, R. Stiefelhagen: Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. JIVP 2008.
[3] Y. Li, C. Huang, R. Nevatia: Learning to associate: HybridBoosted multi-target tracker for crowded scene. CVPR 2009.


eXTReMe Tracker