Multi-View Stereo Network with attention thin volume

• URL: http://arxiv.org/abs/2110.08556v1
• Date: Sat, 16 Oct 2021 11:51:23 GMT
• Title: Multi-View Stereo Network with attention thin volume
• Authors: Zihang Wan
• Abstract summary: We propose an efficient multi-view stereo (MVS) network for infering depth value from multiple RGB images. We introduce the self-attention mechanism to fully aggregate the dominant information from input images. We also introduce the group-wise correlation to feature aggregation, which greatly reduces the memory and calculation burden.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose an efficient multi-view stereo (MVS) network for infering depth value from multiple RGB images. Recent studies have shown that mapping the geometric relationship in real space to neural network is an essential topic of the MVS problem. Specifically, these methods focus on how to express the correspondence between different views by constructing a nice cost volume. In this paper, we propose a more complete cost volume construction approach based on absorbing previous experience. First of all, we introduce the self-attention mechanism to fully aggregate the dominant information from input images and accurately model the long-range dependency, so as to selectively aggregate reference features. Secondly, we introduce the group-wise correlation to feature aggregation, which greatly reduces the memory and calculation burden. Meanwhile, this method enhances the information interaction between different feature channels. With this approach, a more lightweight and efficient cost volume is constructed. Finally we follow the coarse to fine strategy and refine the depth sampling range scale by scale with the help of uncertainty estimation. We further combine the previous steps to get the attention thin volume. Quantitative and qualitative experiments are presented to demonstrate the performance of our model.

Related papers

• Multi-view Aggregation Network for Dichotomous Image Segmentation [76.75904424539543]
  Dichotomous Image (DIS) has recently emerged towards high-precision object segmentation from high-resolution natural images. Existing methods rely on tedious multiple encoder-decoder streams and stages to gradually complete the global localization and local refinement. Inspired by it, we model DIS as a multi-view object perception problem and provide a parsimonious multi-view aggregation network (MVANet) Experiments on the popular DIS-5K dataset show that our MVANet significantly outperforms state-of-the-art methods in both accuracy and speed.
  arXiv  Detail & Related papers  (2024-04-11T03:00:00Z)
• Differentiable Information Bottleneck for Deterministic Multi-view Clustering [9.723389925212567]
  We propose a new differentiable information bottleneck (DIB) method, which provides a deterministic and analytical MVC solution. Specifically, we first propose to directly fit the mutual information of high-dimensional spaces by leveraging normalized kernel Gram matrix. Then, based on the new mutual information measurement, a deterministic multi-view neural network with analytical gradients is explicitly trained to parameterize IB principle.
  arXiv  Detail & Related papers  (2024-03-23T02:13:22Z)
• DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation [50.08080424613603]
  Long-range correlation is essential for accurate monocular depth estimation. We propose to leverage the Transformer to model this global context with an effective attention mechanism. Our proposed model, termed DepthFormer, surpasses state-of-the-art monocular depth estimation methods with prominent margins.
  arXiv  Detail & Related papers  (2022-03-27T05:03:56Z)
• Curvature-guided dynamic scale networks for Multi-view Stereo [10.667165962654996]
  This paper focuses on learning a robust feature extraction network to enhance the performance of matching costs without heavy computation. We present a dynamic scale feature extraction network, namely, CDSFNet. It is composed of multiple novel convolution layers, each of which can select a proper patch scale for each pixel guided by the normal curvature of the image surface.
  arXiv  Detail & Related papers  (2021-12-11T14:41:05Z)
• Adaptive Context-Aware Multi-Modal Network for Depth Completion [107.15344488719322]
  We propose to adopt the graph propagation to capture the observed spatial contexts. We then apply the attention mechanism on the propagation, which encourages the network to model the contextual information adaptively. Finally, we introduce the symmetric gated fusion strategy to exploit the extracted multi-modal features effectively. Our model, named Adaptive Context-Aware Multi-Modal Network (ACMNet), achieves the state-of-the-art performance on two benchmarks.
  arXiv  Detail & Related papers  (2020-08-25T06:00:06Z)
• Sequential Hierarchical Learning with Distribution Transformation for Image Super-Resolution [83.70890515772456]
  We build a sequential hierarchical learning super-resolution network (SHSR) for effective image SR. We consider the inter-scale correlations of features, and devise a sequential multi-scale block (SMB) to progressively explore the hierarchical information. Experiment results show SHSR achieves superior quantitative performance and visual quality to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-19T01:35:53Z)
• Crowd Counting via Hierarchical Scale Recalibration Network [61.09833400167511]
  We propose a novel Hierarchical Scale Recalibration Network (HSRNet) to tackle the task of crowd counting. HSRNet models rich contextual dependencies and recalibrating multiple scale-associated information. Our approach can ignore various noises selectively and focus on appropriate crowd scales automatically.
  arXiv  Detail & Related papers  (2020-03-07T10:06:47Z)
• ADRN: Attention-based Deep Residual Network for Hyperspectral Image Denoising [52.01041506447195]
  We propose an attention-based deep residual network to learn a mapping from noisy HSI to the clean one. Experimental results demonstrate that our proposed ADRN scheme outperforms the state-of-the-art methods both in quantitative and visual evaluations.
  arXiv  Detail & Related papers  (2020-03-04T08:36:27Z)
• Learning Inverse Depth Regression for Multi-View Stereo with Correlation Cost Volume [32.41293572426403]
  Deep learning has shown to be effective for depth inference in multi-view stereo (MVS) However, the scalability and accuracy still remain an open problem in this domain. Inspired by the group-wise correlation in stereo matching, we propose an average group-wise correlation similarity measure to construct a lightweight cost volume.
  arXiv  Detail & Related papers  (2019-12-26T01:40:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.