Deep-Assisted High Resolution Binocular Stereo Depth Reconstruction

Authors: Yaoyu Hu, Weikun Zhen, and Sebastian Scherer.

<img>Cover figure.

This is the project page of Deep-Assisted High Resolution Binocular Stereo Depth Reconstruction.

Here we list the comparisons between PSMNU, SGBM, SGBMUR, and SGBMP on Middleburry Evaluation V3 with the training cases in their full resolution.

Middlebury dataset

In the following materials, the abbreviation GTC means Ground Truth Coverage, it is the percentage of pixels which have un-occluded true disparities. Middlebury dataset does not provide disparities for all of the pixels in the left image. Some pixels are occluded and some pixels have Inf. disparities. GTCs are different across the cases. All pixels that are NOT covered by the ground truth will NOT participate in the calculation of Bad1.0, invalid, avgErr, and stdErr.

The disparity ranges are determined by looking at the true disparity first. Then a total number of disparity dividable by 8 is chosen. Disparity ranges are converted to minDisparity and numDisparity for SGBM, SGBMUR, and SGBMP. All of the true disparities are confined by the disparity ranges.

SGBM has the lowest Bad1.0 among all the cases. This is because SGBM invalidates lots of pixels and only keeps the pixels which it has ‘confident’ about. The same reason leads to SGMB’s high invalid value.

To compare the performance, we have to consider all the metrics together rather than comparing only one of them. The goal of SGBMP is achieving low avgErr and low invalid at the same time and do its job on our 4K stereo images. Due to the intrinsic difficultis in the hard regions for stereo reconstruction, disparity predictions by SGBMP in SGBM’s invalid regions contains additional noise, making the Bad1.0 value higher than SGBM. The increased Bad1.0 is the cost SGBMP paied to have lower invalid. However, SGBMP manages to achieve lower avgErr, lower invalid, and lower stdErr for most of the cases, meaning it does perform better than SGBM.

The 12 cases on which SGBMP has lowest stdErr are: Adirondack, ArtL, Jadeplant*, Motorcycle*, MotorcycleE, PianoL, Pipes, Playroom, Playtable, PlaytableP, Shelves, and Teddy. The cases marked by * are those SGBMP did not get the lowest avgErr.

Adirondack, GTC 91%, disparity range 1-304 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 20.92 (7.52) 1.18 4.81
SGBM 21.59 31.60 9.54 30.54
SGBMUR 32.40 18.85 14.17 37.71
SGBMP 24.82 8.47 1.10 3.61

<img>Adirondack.
Adirondack, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

ArtL, GTC 76%, disparity range 1-256 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 14.77 (9.25) 2.94 10.34
SGBM 9.05 25.00 4.50 17.89
SGBMUR 12.87 18.96 6.92 24.18
SGBMP 11.52 13.44 1.78 7.63

<img>ArtL.
ArtL, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Jadeplant, GTC 74.3%, disparity range 1-640 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 35.09 (19.35) 18.29 54.90
SGBM 13.90 34.26 7.54 37.45
SGBMUR 19.61 27.29 11.62 46.53
SGBMP 26.11 28.23 7.84 35.74

<img>Jadeplant.
Jadeplant, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Motorcycle, GTC 86%, disparity range 1-288 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 29.11 (6.28) 2.95 11.64
SGBM 17.68 13.81 1.95 9.91
SGBMUR 20.21 9.24 2.35 11.30
SGBMP 27.65 8.44 2.09 8.69

<img>Motorcycle.
Motorcycle, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

MotorcycleE, GTC 86%, disparity range 1-288 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 30.99 (6.28) 3.20 12.21
SGBM 17.28 31.93 8.38 29.30
SGBMUR 23.83 21.40 13.16 37.46
SGBMP 34.59 8.93 2.30 8.62

<img>MotorcycleE.
MotorcycleE, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Piano, GTC 86%, disparity range 1-272 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 30.75 (4.49) 3.09 12.99
SGBM 25.78 19.59 4.09 9.51
SGBMUR 34.30 9.23 5.31 11.27
SGBMP 35.90 5.21 3.04 12.66

<img>Piano.
Piano, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

PianoL, GTC 86%, disparity range 1-272 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 59.32 (4.49) 14.56 29.67
SGBM 21.44 44.60 25.01 52.33
SGBMUR 35.69 28.18 34.76 57.11
SGBMP 55.32 10.87 11.92 26.49

<img>PianoL.
PianoL, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Pipes, GTC 78.6%, disparity range 1-304 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 27.92 (5.23) 3.75 13.81
SGBM 13.46 13.94 3.43 16.71
SGBMUR 16.80 9.45 4.26 18.41
SGBMP 21.54 8.37 2.44 10.84

<img>Pipes.
Pipes, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Playroom, GTC 79.9%, disparity range 1-336 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 42.08 (4.55) 4.03 12.46
SGBM 23.62 31.89 5.05 16.27
SGBMUR 36.41 16.17 7.42 20.28
SGBMP 42.56 6.87 3.33 10.48

<img>Pipes.
Playroom, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Playtable, GTC 86%, disparity range 1-304 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 50.96 (6.54) 13.72 31.52
SGBM 31.39 28.35 10.52 28.23
SGBMUR 41.49 15.61 10.85 27.61
SGBMP 42.41 10.85 7.13 19.99

<img>Playtable.
Playtable, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

PlaytableP, GTC 86.2, disparity range 1-304 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 38.00 (6.53) 7.83 22.77
SGBM 17.46 20.79 2.15 7.30
SGBMUR 25.57 10.75 2.78 8.31
SGBMP 25.01 8.65 1.68 5.19

<img>PlaytableP.
PlaytableP, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Recycle, GTC 90.4%, disparity range 1-272 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 29.38 (7.49) 2.51 10.01
SGBM 26.97 25.73 2.31 8.33
SGBMUR 36.58 13.40 3.54 11.73
SGBMP 37.72 8.57 2.43 9.06

<img>Recycle.
Recycle, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Shelves, GTC 82.8%, disparity range 1-240 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 61.72 (5.94) 4.86 12.34
SGBM 25.96 35.95 5.80 14.97
SGBMUR 42.90 17.69 8.31 17.01
SGBMP 50.12 6.91 4.00 9.20

<img>Shelves.
Shelves, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Teddy, GTC 86.8%, disparity range 1-256 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 9.78 (7.97) 1.19 4.17
SGBM 7.94 13.51 1.38 8.37
SGBMUR 9.92 10.51 1.82 10.83
SGBMP 10.85 8.74 1.12 3.49

<img>Teddy.
Teddy, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.

Vintage, GTC 88.2%, disparity range 1-768 pixels

Method Bad1.0 invalid avgErr stdErr
PSMNU 37.54 (23.09) 8.33 27.43
SGBM 23.57 45.02 5.72 14.16
SGBMUR 36.36 29.49 7.91 21.57
SGBMP 33.88 24.90 6.48 20.49

<img>Vintage.
Vintage, true disparity, PSMNU (up-sampled), SGBM, SGBMUR, SGBMP.