Method

Learning to Track: Online Multi-Object Tracking by Decision Making [on] [MDP]
http://cvgl.stanford.edu/projects/MDP_tracking

Submitted on 1 Feb. 2016 05:18 by
Yu Xiang (Stanford University)

Running time:0.9 s
Environment:8 cores @ 3.5 Ghz (Matlab + C/C++)

Method Description:
Online Multi-Object Tracking (MOT) has wide
applications in time-critical video analysis
scenarios, such as robot navigation and
autonomous driving. In tracking-by-detection, a
major challenge of online MOT is how to
robustly
associate noisy object detections on a new
video
frame with previously tracked objects. In this
work, we formulate the online MOT problem as
decision making in Markov Decision Processes
(MDPs), where the lifetime of an object is
modeled with a MDP. Learning a similarity
function for data association is equivalent to
learning a policy for the MDP, and the policy
learning is approached in a reinforcement
learning fashion which benefits from both
advantages of offline-learning and online-
learning for data association. Moreover, our
framework can naturally handle the birth/death
and appearance/disappearance of targets by
treating them as state transitions in the MDP
while leveraging existing online single object
tracking methods.
Parameters:
Use detetions from the SubCNN method evaluated on
the KITTI object detection benchmark
Latex Bibtex:
@inproceedings{Xiang2015ICCV,
author = {Xiang, Yu and Alahi, Alexandre
and
Savarese, Silvio},
title = {Learning to Track: Online Multi-
Object Tracking by Decision Making},
booktitle = {International Conference on
Computer Vision (ICCV)},
pages = {4705--4713},
year = {2015}
}
@inproceedings{xiang2017subcategory,
author = {Xiang, Yu and Choi, Wongun and Lin, Yuanqing
and Savarese, Silvio},
title = {Subcategory-aware Convolutional Neural
Networks for Object Proposals and Detection},
booktitle = {IEEE Winter Conference on Applications of
Computer Vision (WACV)},
year = {2017}
}

Detailed Results

From all 29 test sequences, our benchmark computes the HOTA tracking metrics (HOTA, DetA, AssA, DetRe, DetPr, AssRe, AssPr, LocA) [1] as well as the CLEARMOT, MT/PT/ML, identity switches, and fragmentation [2,3] metrics. The tables below show all of these metrics.


Benchmark HOTA DetA AssA DetRe DetPr AssRe AssPr LocA
CAR 64.79 % 63.04 % 67.05 % 66.18 % 82.22 % 69.61 % 85.61 % 84.24 %
PEDESTRIAN 42.76 % 39.23 % 47.13 % 43.83 % 63.02 % 50.91 % 71.04 % 75.15 %

Benchmark TP FP FN
CAR 27022 7370 661
PEDESTRIAN 13600 9550 2502

Benchmark MOTA MOTP MODA IDSW sMOTA
CAR 76.08 % 82.22 % 76.65 % 196 62.11 %
PEDESTRIAN 47.02 % 70.17 % 47.94 % 213 29.50 %

Benchmark MT rate PT rate ML rate FRAG
CAR 51.54 % 34.15 % 14.31 % 281
PEDESTRIAN 25.77 % 45.70 % 28.52 % 738

Benchmark # Dets # Tracks
CAR 27683 725
PEDESTRIAN 16102 394

This table as LaTeX


This figure as: png pdf

This figure as: png pdf

[1] J. Luiten, A. Os̆ep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, B. Leibe: HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. IJCV 2020.
[2] K. Bernardin, R. Stiefelhagen: Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. JIVP 2008.
[3] Y. Li, C. Huang, R. Nevatia: Learning to associate: HybridBoosted multi-target tracker for crowded scene. CVPR 2009.


eXTReMe Tracker