Method

Language Guidance to Achieve Multi-Modal 3D Multi-Object Tracking [LG3MOT]


Submitted on 24 Feb. 2025 07:33 by
longhui hu (USTB)

Running time:0.26 s
Environment:GPU @ 2.5 Ghz (Python)

Method Description:
The proposed Language-Guided Multi-Modal 3D Multi-
Object Tracking (LG-MM3DMOT) framework integrates
Vision-Language Models (VLMs) to enhance tracking by
aligning image regions with textual concepts using
RegionCLIP. It introduces the Target Semantic
Matching Module (TSM) to filter noisy regions, the
3D Feature EMA Module for temporal feature fusion,
and the Gaussian Confidence Fusion Module for
weighted trajectory confidence computation.
Additionally, the Early Drop Strategy leverages
semantic information to efficiently manage
trajectories by terminating mismatched ones early.
These components collectively improve tracking
accuracy and robustness in complex scenarios.
Parameters:
confidence_thresh=0.1
confidence_his_max=16
max_age=48
Latex Bibtex:

Detailed Results

From all 29 test sequences, our benchmark computes the HOTA tracking metrics (HOTA, DetA, AssA, DetRe, DetPr, AssRe, AssPr, LocA) [1] as well as the CLEARMOT, MT/PT/ML, identity switches, and fragmentation [2,3] metrics. The tables below show all of these metrics.


Benchmark HOTA DetA AssA DetRe DetPr AssRe AssPr LocA
CAR 78.72 % 74.59 % 83.69 % 83.00 % 81.49 % 87.47 % 89.87 % 87.64 %

Benchmark TP FP FN
CAR 32348 2044 2684

Benchmark MOTA MOTP MODA IDSW sMOTA
CAR 86.15 % 86.26 % 86.25 % 35 73.23 %

Benchmark MT rate PT rate ML rate FRAG
CAR 82.46 % 15.08 % 2.46 % 411

Benchmark # Dets # Tracks
CAR 35032 939

This table as LaTeX

This figure as: png pdf

[1] J. Luiten, A. Os̆ep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, B. Leibe: HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. IJCV 2020.
[2] K. Bernardin, R. Stiefelhagen: Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. JIVP 2008.
[3] Y. Li, C. Huang, R. Nevatia: Learning to associate: HybridBoosted multi-target tracker for crowded scene. CVPR 2009.


eXTReMe Tracker