The KITTI Vision Benchmark Suite

Result is zero-shot, cross-dataset transfer (never
trained on KITTI).

We present MSeg, a composite dataset that unifies
semantic segmentation datasets from different
domains. A naive merge of the constituent datasets
yields poor performance due to inconsistent
taxonomies and annotation practices. We reconcile
the taxonomies and bring the pixel-level annotations
into alignment by relabeling more than 220,000
object masks in more than 80,000 images, requiring
more than 1.34 years of collective annotator effort.
The resulting composite dataset enables training a
single semantic segmentation model that functions
effectively across domains and generalizes to
datasets that were not seen during training. We
adopt zero-shot cross-dataset transfer as a bench-
mark to systematically evaluate a model’s robustness
and show that MSeg training yields substantially
more robust models in comparison to training on
individual datasets or naive mixing of datasets
without the presented contributions.

Trained for 3 million crops @ 1080p. Inference is at
360p.

@InProceedings{MSeg_2020_CVPR,
author = {Lambert, John and Liu, Zhuang and Sener,
Ozan and Hays, James and Koltun, Vladlen},
title = {{MSeg}: A Composite Dataset for Multi-
domain Semantic Segmentation},
booktitle = {Computer Vision and Pattern Recognition
(CVPR)},
year = {2020}
}