Neural Surface Reconstruction from Sparse Views Using Epipolar Geometry

• URL: http://arxiv.org/abs/2406.04301v1
• Date: Thu, 6 Jun 2024 17:47:48 GMT
• Title: Neural Surface Reconstruction from Sparse Views Using Epipolar Geometry
• Authors: Kaichen Zhou,
• Abstract summary: We present a novel approach, named EpiS, that incorporates Epipolar information into the reconstruction process. Our method aggregates coarse information from the cost volume into Epipolar features extracted from multiple source views. To address the information gaps in sparse conditions, we integrate depth information from monocular depth estimation using global and local regularization techniques.
• Score: 4.659427498118277
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper addresses the challenge of reconstructing surfaces from sparse view inputs, where ambiguity and occlusions due to missing information pose significant hurdles. We present a novel approach, named EpiS, that incorporates Epipolar information into the reconstruction process. Existing methods in sparse-view neural surface learning have mainly focused on mean and variance considerations using cost volumes for feature extraction. In contrast, our method aggregates coarse information from the cost volume into Epipolar features extracted from multiple source views, enabling the generation of fine-grained Signal Distance Function (SDF)-aware features. Additionally, we employ an attention mechanism along the line dimension to facilitate feature fusion based on the SDF feature. Furthermore, to address the information gaps in sparse conditions, we integrate depth information from monocular depth estimation using global and local regularization techniques. The global regularization utilizes a triplet loss function, while the local regularization employs a derivative loss function. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods, especially in cases with sparse and generalizable conditions.

Related papers

• DiHuR: Diffusion-Guided Generalizable Human Reconstruction [51.31232435994026]
  We introduce DiHuR, a Diffusion-guided model for generalizable Human 3D Reconstruction and view synthesis from sparse, minimally overlapping images. Our method integrates two key priors in a coherent manner: the prior from generalizable feed-forward models and the 2D diffusion prior, and it requires only multi-view image training, without 3D supervision.
  arXiv  Detail & Related papers  (2024-11-16T03:52:23Z)
• Sparse Modelling for Feature Learning in High Dimensional Data [0.0]
  This paper presents an innovative approach to dimensionality reduction and feature extraction in high-dimensional datasets. The proposed framework integrates sparse modeling techniques into a comprehensive pipeline for efficient and interpretable feature selection. We aim to advance the understanding and application of sparse modeling in machine learning, particularly in the context of wood surface defect detection.
  arXiv  Detail & Related papers  (2024-09-28T14:17:59Z)
• Generalizable Non-Line-of-Sight Imaging with Learnable Physical Priors [52.195637608631955]
  Non-line-of-sight (NLOS) imaging has attracted increasing attention due to its potential applications. Existing NLOS reconstruction approaches are constrained by the reliance on empirical physical priors. We introduce a novel learning-based solution, comprising two key designs: Learnable Path Compensation (LPC) and Adaptive Phasor Field (APF)
  arXiv  Detail & Related papers  (2024-09-21T04:39:45Z)
• FNP: Fourier Neural Processes for Arbitrary-Resolution Data Assimilation [58.149902193341816]
  We propose textittextbfFourier Neural Processes (FNP) for textitarbitrary-resolution data assimilation in this paper. Our FNP trained on a fixed resolution can directly handle the assimilation of observations with out-of-distribution resolutions and the observational information reconstruction task without additional fine-tuning.
  arXiv  Detail & Related papers  (2024-06-03T12:24:24Z)
• DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery [71.6345505427213]
  DPMesh is an innovative framework for occluded human mesh recovery. It capitalizes on the profound diffusion prior about object structure and spatial relationships embedded in a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-04-01T18:59:13Z)
• Subspace-Guided Feature Reconstruction for Unsupervised Anomaly Localization [5.085309164633571]
  Unsupervised anomaly localization plays a critical role in industrial manufacturing. Most recent methods perform feature matching or reconstruction for the target sample with pre-trained deep neural networks. We propose a novel subspace-guided feature reconstruction framework to pursue adaptive feature approximation for anomaly localization.
  arXiv  Detail & Related papers  (2023-09-25T06:58:57Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• RecFNO: a resolution-invariant flow and heat field reconstruction method from sparse observations via Fourier neural operator [8.986743262828009]
  We propose an end-to-end physical field reconstruction method with both excellent performance and mesh transferability named RecFNO. The proposed method aims to learn the mapping from sparse observations to flow and heat field in infinite-dimensional space. The experiments conducted on fluid mechanics and thermology problems show that the proposed method outperforms existing POD-based and CNN-based methods in most cases.
  arXiv  Detail & Related papers  (2023-02-20T07:20:22Z)
• Progressive Spatio-Temporal Bilinear Network with Monte Carlo Dropout for Landmark-based Facial Expression Recognition with Uncertainty Estimation [93.73198973454944]
  The performance of our method is evaluated on three widely used datasets. It is comparable to that of video-based state-of-the-art methods while it has much less complexity.
  arXiv  Detail & Related papers  (2021-06-08T13:40:30Z)
• Improving Monocular Depth Estimation by Leveraging Structural Awareness and Complementary Datasets [21.703238902823937]
  We propose a structure-aware neural network with spatial attention blocks to exploit the spatial relationship of visual features. Second, we introduce a global focal relative loss for uniform point pairs to enhance spatial constraint in the prediction. Third, based on analysis of failure cases for prior methods, we collect a new Hard Case (HC) Depth dataset of challenging scenes.
  arXiv  Detail & Related papers  (2020-07-22T08:21:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.