Neural Disparity Refinement for Arbitrary Resolution Stereo

• URL: http://arxiv.org/abs/2110.15367v1
• Date: Thu, 28 Oct 2021 18:00:00 GMT
• Title: Neural Disparity Refinement for Arbitrary Resolution Stereo
• Authors: Filippo Aleotti, Fabio Tosi, Pierluigi Zama Ramirez, Matteo Poggi, Samuele Salti, Stefano Mattoccia, Luigi Di Stefano
• Abstract summary: We introduce a novel architecture for neural disparity refinement aimed at facilitating deployment of 3D computer vision on cheap and widespread consumer devices. Our approach relies on a continuous formulation that enables to estimate a refined disparity map at any arbitrary output resolution.
• Score: 67.55946402652778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce a novel architecture for neural disparity refinement aimed at facilitating deployment of 3D computer vision on cheap and widespread consumer devices, such as mobile phones. Our approach relies on a continuous formulation that enables to estimate a refined disparity map at any arbitrary output resolution. Thereby, it can handle effectively the unbalanced camera setup typical of nowadays mobile phones, which feature both high and low resolution RGB sensors within the same device. Moreover, our neural network can process seamlessly the output of a variety of stereo methods and, by refining the disparity maps computed by a traditional matching algorithm like SGM, it can achieve unpaired zero-shot generalization performance compared to state-of-the-art end-to-end stereo models.

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• SD-MVS: Segmentation-Driven Deformation Multi-View Stereo with Spherical Refinement and EM optimization [6.886220026399106]
  We introduce Multi-View Stereo (SD-MVS) to tackle challenges in 3D reconstruction of textureless areas. We are the first to adopt the Segment Anything Model (SAM) to distinguish semantic instances in scenes. We propose a unique refinement strategy that combines spherical coordinates and gradient descent on normals and pixelwise search interval on depths.
  arXiv  Detail & Related papers  (2024-01-12T05:25:57Z)
• Unifying Flow, Stereo and Depth Estimation [121.54066319299261]
  We present a unified formulation and model for three motion and 3D perception tasks. We formulate all three tasks as a unified dense correspondence matching problem. Our model naturally enables cross-task transfer since the model architecture and parameters are shared across tasks.
  arXiv  Detail & Related papers  (2022-11-10T18:59:54Z)
• DiffuStereo: High Quality Human Reconstruction via Diffusion-based Stereo Using Sparse Cameras [33.6247548142638]
  We propose DiffuStereo, a novel system using only sparse cameras for high-quality 3D human reconstruction. At its core is a novel diffusion-based stereo module, which introduces diffusion models into the iterative stereo matching network. We present a multi-level stereo network architecture to handle high-resolution (up to 4k) inputs without requiring unaffordable memory footprint.
  arXiv  Detail & Related papers  (2022-07-16T19:08:18Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)
• SGM3D: Stereo Guided Monocular 3D Object Detection [62.11858392862551]
  We propose a stereo-guided monocular 3D object detection network, termed SGM3D. We exploit robust 3D features extracted from stereo images to enhance the features learned from the monocular image. Our method can be integrated into many other monocular approaches to boost performance without introducing any extra computational cost.
  arXiv  Detail & Related papers  (2021-12-03T13:57:14Z)
• SMD-Nets: Stereo Mixture Density Networks [68.56947049719936]
  We propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning framework compatible with a wide class of 2D and 3D architectures. Specifically, we exploit bimodal mixture densities as output representation and show that this allows for sharp and precise disparity estimates near discontinuities. We carry out comprehensive experiments on a new high-resolution and highly realistic synthetic stereo dataset, consisting of stereo pairs at 8Mpx resolution, as well as on real-world stereo datasets.
  arXiv  Detail & Related papers  (2021-04-08T16:15:46Z)
• Instantaneous Stereo Depth Estimation of Real-World Stimuli with a Neuromorphic Stereo-Vision Setup [4.28479274054892]
  Spiking Neural Network (SNN) architectures for stereo vision have the potential of simplifying the stereo-matching problem. We validate a brain-inspired event-based stereo-matching architecture implemented on a mixed-signal neuromorphic processor with real-world data.
  arXiv  Detail & Related papers  (2021-04-06T14:31:23Z)
• Du$^2$Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels [16.797169907541164]
  We present a novel approach based on neural networks for depth estimation that combines stereo from dual cameras with stereo from a dual-pixel sensor. Our network uses a novel architecture to fuse these two sources of information and can overcome the limitations of pure binocular stereo matching.
  arXiv  Detail & Related papers  (2020-03-31T15:39:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.