Method

ReMOTS: Self-Supervised Refining Multi-Object Tracking and Segmentation [ReMOTS]


Submitted on 25 Aug. 2020 15:22 by
Fan Yang (Nara Institute of Science and Technology)

Running time:3 s
Environment:1 core @ 2.5 Ghz (Python)

Method Description:
We apply ReMOTS on public detections.
The confidence threshold of public detection are:
Pedestrian: 0.2
Car: 0.5

The process is:
(1) Training the appearance encoder using predicted
masks.
(2) Associating observations across adjacent frames
to form short-term tracklets.
(3) Training the appearance encoder using short-
term tracklets as reliable pseudo labels.
(4) Merging short-term tracklets to long-term
tracklets utilizing adopted appearance features and
thresholds that are automatically obtained from
statistical information.

For more details please refer to our paper.
Parameters:
N
Latex Bibtex:
@misc{yang2020remots,
title={ReMOTS: Self-Supervised Refining Multi-
Object Tracking and Segmentation},
author={Fan Yang and Xin Chang and Chenyu Dang
and Ziqiang Zheng and Sakriani Sakti and Satoshi
Nakamura and Yang Wu},
year={2020},
eprint={2007.03200},
archivePrefix={arXiv},
primaryClass={cs.CV}
}

Detailed Results

From all 29 test sequences, our benchmark computes the commonly used tracking metrics (adapted for the segmentation case): CLEARMOT, MT/PT/ML, identity switches, and fragmentations [1,2]. The tables below show all of these metrics.


Benchmark sMOTSA MOTSA MOTSP MODSA MODSP
CAR 75.90 % 86.70 % 88.20 % 88.70 % 90.70 %
PEDESTRIAN 66.00 % 81.30 % 82.00 % 83.20 % 94.00 %

Benchmark recall precision F1 TP FP FN FAR #objects #trajectories
CAR 92.10 % 96.40 % 94.20 % 33856 1257 2904 11.30 % 53256 3853
PEDESTRIAN 85.30 % 97.60 % 91.00 % 17655 434 3042 3.90 % 24335 1425

Benchmark MT PT ML IDS FRAG
CAR 84.50 % 14.90 % 0.60 % 716 905
PEDESTRIAN 62.60 % 31.90 % 5.60 % 391 551

This table as LaTeX


[1] K. Bernardin, R. Stiefelhagen: Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. JIVP 2008.
[2] Y. Li, C. Huang, R. Nevatia: Learning to associate: HybridBoosted multi-target tracker for crowded scene. CVPR 2009.


eXTReMe Tracker