Method

RecResNet: A Recurrent Residual CNN Architecture for Disparity Map Enhancement [RecResNet]
https://github.com/kbatsos/RecResNet

Submitted on 1 Mar. 2019 19:36 by
KONSTANTINOS BATSOS (Stevens Institute of technology)

Running time:0.3 s
Environment:GPU @ NVIDIA TITAN X (Tensorflow)

Method Description:
We present a neural network architecture applied to
the
problem of refining a dense disparity map generated
by
a stereo algorithm to which we have no access. Our
approach is able to learn which disparity values
should be
modified and how, from a training set of images,
estimated
disparity maps and the corresponding ground truth.
Its only
input at test time is a disparity map and the
reference image. Two design characteristics are
critical for the success
of our network: (i) it is formulated as a recurrent
neural
network, and (ii) it estimates the output refined
disparity
map as a combination of residuals computed at
multiple
scales, that is at different up-sampling and down-
sampling
rates. The first property allows the network, which
we named
RecResNet, to progressively improve the disparity
map, while
the second property allows the corrections to come
from
different scales of analysis, addressing different
types of
errors in the current disparity map. We present
competitive quantitative and qualitative results on
the KITTI 2012
and 2015 benchmarks that surpass the accuracy of
previous disparity refinement methods.
Parameters:
See our paper
Latex Bibtex:
@inproceedings{batsos2018recresnet,
title={RecResNet: A Recurrent Residual CNN
Architecture for Disparity Map Enhancement},
author={Batsos, Konstantinos and Mordohai,
Philipos},
booktitle={ In International Conference on 3D
Vision (3DV) },
year={2018}
}

Detailed Results

This page provides detailed results for the method(s) selected. For each of the first 20 test images, the number of erroneous pixels at all thresholds is depicted in the table. Underneath, the left input image, the disparity / end-point error map and the estimated (and interpolated) disparity / optical flow map are shown. The error map scales linearly between 0 (black) and >=5 (white) pixels error. Red denotes all occluded pixels, falling outside the image boundaries. The false color map is scaled to the largest ground truth disparity / flow value.

Test Set Average

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 3.37 % 4.38 % 0.6 px 0.7 px
3 pixels 2.21 % 2.94 % 0.6 px 0.7 px
4 pixels 1.73 % 2.31 % 0.6 px 0.7 px
5 pixels 1.45 % 1.92 % 0.6 px 0.7 px
This table as LaTeX

Reflective Regions

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 21.60 % 24.78 % 2.1 px 2.5 px
3 pixels 14.25 % 17.32 % 2.1 px 2.5 px
4 pixels 10.64 % 13.40 % 2.1 px 2.5 px
5 pixels 8.52 % 10.98 % 2.1 px 2.5 px
This table as LaTeX

Test Image 0

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 3.23 % 3.67 % 0.6 px 0.6 px
3 pixels 2.09 % 2.33 % 0.6 px 0.6 px
4 pixels 1.52 % 1.71 % 0.6 px 0.6 px
5 pixels 1.26 % 1.44 % 0.6 px 0.6 px
This table as LaTeX





Test Image 1

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 3.24 % 5.38 % 0.5 px 0.6 px
3 pixels 1.94 % 2.99 % 0.5 px 0.6 px
4 pixels 1.41 % 2.16 % 0.5 px 0.6 px
5 pixels 1.21 % 1.71 % 0.5 px 0.6 px
This table as LaTeX





Test Image 2

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.22 % 1.44 % 0.4 px 0.4 px
3 pixels 0.82 % 0.86 % 0.4 px 0.4 px
4 pixels 0.69 % 0.73 % 0.4 px 0.4 px
5 pixels 0.57 % 0.61 % 0.4 px 0.4 px
This table as LaTeX





Test Image 3

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.83 % 3.40 % 0.4 px 0.5 px
3 pixels 0.59 % 1.73 % 0.4 px 0.5 px
4 pixels 0.33 % 1.13 % 0.4 px 0.5 px
5 pixels 0.23 % 0.76 % 0.4 px 0.5 px
This table as LaTeX





Test Image 4

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.92 % 2.86 % 0.4 px 0.5 px
3 pixels 1.10 % 1.35 % 0.4 px 0.5 px
4 pixels 1.01 % 1.23 % 0.4 px 0.5 px
5 pixels 0.90 % 1.12 % 0.4 px 0.5 px
This table as LaTeX





Test Image 5

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.27 % 1.36 % 0.4 px 0.4 px
3 pixels 0.76 % 0.82 % 0.4 px 0.4 px
4 pixels 0.57 % 0.62 % 0.4 px 0.4 px
5 pixels 0.49 % 0.51 % 0.4 px 0.4 px
This table as LaTeX





Test Image 6

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.51 % 5.02 % 0.8 px 0.8 px
3 pixels 2.35 % 2.72 % 0.8 px 0.8 px
4 pixels 1.66 % 2.01 % 0.8 px 0.8 px
5 pixels 1.42 % 1.73 % 0.8 px 0.8 px
This table as LaTeX





Test Image 7

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 5.54 % 6.77 % 0.9 px 1.0 px
3 pixels 3.47 % 4.55 % 0.9 px 1.0 px
4 pixels 2.45 % 3.43 % 0.9 px 1.0 px
5 pixels 1.86 % 2.76 % 0.9 px 1.0 px
This table as LaTeX





Test Image 8

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.41 % 1.86 % 0.4 px 0.5 px
3 pixels 0.88 % 1.34 % 0.4 px 0.5 px
4 pixels 0.74 % 1.20 % 0.4 px 0.5 px
5 pixels 0.67 % 1.13 % 0.4 px 0.5 px
This table as LaTeX





Test Image 9

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 6.04 % 8.28 % 0.9 px 1.2 px
3 pixels 4.21 % 6.37 % 0.9 px 1.2 px
4 pixels 3.31 % 5.31 % 0.9 px 1.2 px
5 pixels 2.63 % 4.42 % 0.9 px 1.2 px
This table as LaTeX





Test Image 10

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.60 % 2.51 % 0.4 px 0.5 px
3 pixels 0.93 % 1.55 % 0.4 px 0.5 px
4 pixels 0.77 % 1.13 % 0.4 px 0.5 px
5 pixels 0.70 % 0.91 % 0.4 px 0.5 px
This table as LaTeX





Test Image 11

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 3.81 % 5.51 % 0.7 px 0.8 px
3 pixels 2.43 % 2.80 % 0.7 px 0.8 px
4 pixels 1.75 % 1.86 % 0.7 px 0.8 px
5 pixels 1.44 % 1.47 % 0.7 px 0.8 px
This table as LaTeX





Test Image 12

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.57 % 6.03 % 1.0 px 1.1 px
3 pixels 3.61 % 4.59 % 1.0 px 1.1 px
4 pixels 3.21 % 3.92 % 1.0 px 1.1 px
5 pixels 3.03 % 3.50 % 1.0 px 1.1 px
This table as LaTeX





Test Image 13

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.52 % 5.77 % 0.9 px 1.0 px
3 pixels 3.34 % 4.51 % 0.9 px 1.0 px
4 pixels 2.77 % 3.84 % 0.9 px 1.0 px
5 pixels 2.41 % 3.35 % 0.9 px 1.0 px
This table as LaTeX





Test Image 14

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 0.83 % 1.71 % 0.4 px 0.4 px
3 pixels 0.37 % 0.83 % 0.4 px 0.4 px
4 pixels 0.27 % 0.41 % 0.4 px 0.4 px
5 pixels 0.23 % 0.30 % 0.4 px 0.4 px
This table as LaTeX





Test Image 15

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 3.28 % 3.56 % 0.5 px 0.6 px
3 pixels 1.87 % 1.86 % 0.5 px 0.6 px
4 pixels 1.27 % 1.24 % 0.5 px 0.6 px
5 pixels 0.88 % 0.86 % 0.5 px 0.6 px
This table as LaTeX





Test Image 16

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 2.81 % 2.77 % 0.6 px 0.6 px
3 pixels 1.37 % 1.34 % 0.6 px 0.6 px
4 pixels 0.82 % 0.81 % 0.6 px 0.6 px
5 pixels 0.58 % 0.57 % 0.6 px 0.6 px
This table as LaTeX





Test Image 17

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 4.91 % 6.11 % 0.7 px 0.8 px
3 pixels 3.60 % 4.66 % 0.7 px 0.8 px
4 pixels 2.82 % 3.68 % 0.7 px 0.8 px
5 pixels 2.34 % 2.91 % 0.7 px 0.8 px
This table as LaTeX





Test Image 18

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 0.90 % 1.28 % 0.3 px 0.4 px
3 pixels 0.52 % 0.76 % 0.3 px 0.4 px
4 pixels 0.35 % 0.46 % 0.3 px 0.4 px
5 pixels 0.28 % 0.35 % 0.3 px 0.4 px
This table as LaTeX





Test Image 19

Error Out-Noc Out-All Avg-Noc Avg-All
2 pixels 1.68 % 1.80 % 0.5 px 0.5 px
3 pixels 0.89 % 0.86 % 0.5 px 0.5 px
4 pixels 0.74 % 0.72 % 0.5 px 0.5 px
5 pixels 0.66 % 0.64 % 0.5 px 0.5 px
This table as LaTeX







eXTReMe Tracker