Method

ReMOTS: Self-Supervised Refining Multi-Object Tracking and Segmentation [ReMOTS]


Submitted on 25 Aug. 2020 15:22 by
Fan Yang (Nara Institute of Science and Technology)

Running time:3 s
Environment:1 core @ 2.5 Ghz (Python)

Method Description:
We apply ReMOTS on public detections.
The confidence threshold of public detection are:
Pedestrian: 0.2
Car: 0.5

The process is:
(1) Training the appearance encoder using predicted
masks.
(2) Associating observations across adjacent frames
to form short-term tracklets.
(3) Training the appearance encoder using short-
term tracklets as reliable pseudo labels.
(4) Merging short-term tracklets to long-term
tracklets utilizing adopted appearance features and
thresholds that are automatically obtained from
statistical information.

For more details please refer to our paper.
Parameters:
N
Latex Bibtex:
@misc{yang2020remots,
title={ReMOTS: Self-Supervised Refining Multi-
Object Tracking and Segmentation},
author={Fan Yang and Xin Chang and Chenyu Dang
and Ziqiang Zheng and Sakriani Sakti and Satoshi
Nakamura and Yang Wu},
year={2020},
eprint={2007.03200},
archivePrefix={arXiv},
primaryClass={cs.CV}
}

Detailed Results

From all 29 test sequences, our benchmark computes the HOTA tracking metrics (HOTA, DetA, AssA, DetRe, DetPr, AssRe, AssPr, LocA) [1] as well as the CLEARMOT, MT/PT/ML, identity switches, and fragmentation [2,3] metrics. The tables below show all of these metrics.


Benchmark HOTA DetA AssA DetRe DetPr AssRe AssPr LocA
CAR 71.61 % 78.32 % 65.98 % 83.51 % 87.42 % 68.03 % 92.61 % 89.33 %
PEDESTRIAN 58.81 % 67.96 % 52.38 % 71.86 % 82.22 % 54.40 % 88.23 % 84.18 %

Benchmark TP FP FN
CAR 33858 2902 1255
PEDESTRIAN 17657 3040 432

Benchmark MOTSA MOTSP MODSA IDSW sMOTSA
CAR 86.74 % 88.25 % 88.69 % 716 75.92 %
PEDESTRIAN 81.33 % 82.00 % 83.22 % 392 65.97 %

Benchmark MT rate PT rate ML rate FRAG
CAR 84.53 % 14.87 % 0.60 % 713
PEDESTRIAN 62.59 % 31.85 % 5.56 % 567

Benchmark # Dets # Tracks
CAR 35113 1804
PEDESTRIAN 18089 743

This table as LaTeX


This figure as: png pdf

This figure as: png pdf

[1] J. Luiten, A. Os̆ep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, B. Leibe: HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. IJCV 2020.
[2] K. Bernardin, R. Stiefelhagen: Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics. JIVP 2008.
[3] Y. Li, C. Huang, R. Nevatia: Learning to associate: HybridBoosted multi-target tracker for crowded scene. CVPR 2009.


eXTReMe Tracker