The KITTI Vision Benchmark Suite

Camera and lidar are important sensor modalities for
robotics in general and self-driving cars in particular. The
sensors provide complementary information offering an
opportunity for tight sensor-fusion. Surprisingly, lidar-only
methods outperform fusion methods on the main
benchmark datasets, suggesting a gap in the literature. In
this work, we propose PointPainting: a sequential fusion
method to fill this gap. PointPainting works by projecting
lidar points into the output of an image-only semantic
segmentation network and appending the class scores to
each point. The appended (painted) point cloud can then be
fed to any lidar-only method. Experiments show large
improvements on three different state-of-the art methods,
Point-RCNN, VoxelNet and PointPillars on the KITTI and
nuScenes datasets. The painted version of PointRCNN
represents a new state of the art on the KITTI leaderboard
for the bird's-eye view detection task. In ablation, we study
how the effects of Painting depends on the quality and
format of the semantic segmentation output, and
demonstrate how latency can be minimized through
pipelining.

@article{vora2019pointpainting,
title={PointPainting: Sequential Fusion for 3D Object
Detection},
author={Vora, Sourabh and Lang, Alex H and Helou, Bassam
and Beijbom, Oscar},
journal={CVPR},
year={2020}
}