VA-DepthNet: A Variational Approach to Single Image Depth Prediction

Ce Liu1
Suryansh Kumar*1
Shuhang Gu2
Radu Timofte1, 3
Luc Van Gool1, 4
CVL ETH Zürich1, UESTC China2, University of Würzburg3, KU Lueven4
International Conference on Learning Representations (ICLR), 2023.

(Spotlight Oral Presentation)





Abstract
We introduce VA-DepthNet, a simple, effective, and accurate deep neural network approach for the single-image depth prediction (SIDP) problem. The proposed approach advocates using classical first-order variational constraints for this problem. While state-of-the-art deep neural network methods for SIDP learn the scene depth from images in a supervised setting, they often overlook the invaluable invariances and priors in the rigid scene space, such as the regularity of the scene. The paper's main contribution is to reveal the benefit of classical and well-founded variational constraints in the neural network design for the SIDP task. It is shown that imposing first-order variational constraints in the scene space together with popular encoder-decoder-based network architecture design provides excellent results for the supervised SIDP task. The imposed first-order variational constraint makes the network aware of the depth gradient in the scene space, i.e., regularity. The paper demonstrates the usefulness of the proposed approach via extensive evaluation and ablation analysis over several benchmark datasets, such as KITTI, NYU Depth V2, and SUN RGB-D. The VA-DepthNet at test time shows considerable improvements in depth prediction accuracy compared to the prior art and is accurate also at high-frequency regions in the scene space. At the time of writing this paper, our method---labeled as VA-DepthNet, when tested on the KITTI depth-prediction evaluation set benchmarks, shows state-of-the-art results, and is the top-performing published approach.




Paper

VA-DepthNet: A Variational Approach to Single Image Depth Prediction

Ce Liu, Suryansh Kumar*, Shuhang Gu, Radu Timofte, Luc Van Gool.

ICLR 2023, Kigali, Rwanda. (Spotlight Oral Presentation)

[Paper]
[Code]
[ICLR23Link]
[Bibtex]


Result Presentation





ICLR 2023 Oral In Person Presentation





Authors

Ce Liu

Suryansh Kumar*

Shuhang Gu

Radu Timofte

Luc Van Gool


Acknowledgements

This work was partly supported by ETH General Fund (OK), Chinese Scholarship Council (CSC), and The Alexander von Humboldt Foundation.