Visibility-Aware Pixelwise View Selection for Multi-View Stereo Matching
- URL: http://arxiv.org/abs/2302.07182v1
- Date: Tue, 14 Feb 2023 16:50:03 GMT
- Title: Visibility-Aware Pixelwise View Selection for Multi-View Stereo Matching
- Authors: Zhentao Huang, Yukun Shi, Minglun Gong
- Abstract summary: We propose a novel visibility-guided pixelwise view selection scheme.
It progressively refines the set of source views to be used for each pixel in the reference view.
In addition, the Artificial Multi-Bee Colony algorithm is employed to search for optimal solutions for different pixels in parallel.
- Score: 9.915386906818485
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The performance of PatchMatch-based multi-view stereo algorithms depends
heavily on the source views selected for computing matching costs. Instead of
modeling the visibility of different views, most existing approaches handle
occlusions in an ad-hoc manner. To address this issue, we propose a novel
visibility-guided pixelwise view selection scheme in this paper. It
progressively refines the set of source views to be used for each pixel in the
reference view based on visibility information provided by already validated
solutions. In addition, the Artificial Multi-Bee Colony (AMBC) algorithm is
employed to search for optimal solutions for different pixels in parallel.
Inter-colony communication is performed both within the same image and among
different images. Fitness rewards are added to validated and propagated
solutions, effectively enforcing the smoothness of neighboring pixels and
allowing better handling of textureless areas. Experimental results on the DTU
dataset show our method achieves state-of-the-art performance among
non-learning-based methods and retrieves more details in occluded and
low-textured regions.
Related papers
- A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding [76.44979557843367]
We propose a novel multi-view stereo (MVS) framework that gets rid of the depth range prior.
We introduce a Multi-view Disparity Attention (MDA) module to aggregate long-range context information.
We explicitly estimate the quality of the current pixel corresponding to sampled points on the epipolar line of the source image.
arXiv Detail & Related papers (2024-11-04T08:50:16Z) - CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition [73.51329037954866]
We propose a robust global representation method with cross-image correlation awareness for visual place recognition.
Our method uses the attention mechanism to correlate multiple images within a batch.
Our method outperforms state-of-the-art methods by a large margin with significantly less training time.
arXiv Detail & Related papers (2024-02-29T15:05:11Z) - Layered Rendering Diffusion Model for Zero-Shot Guided Image Synthesis [60.260724486834164]
This paper introduces innovative solutions to enhance spatial controllability in diffusion models reliant on text queries.
We present two key innovations: Vision Guidance and the Layered Rendering Diffusion framework.
We apply our method to three practical applications: bounding box-to-image, semantic mask-to-image and image editing.
arXiv Detail & Related papers (2023-11-30T10:36:19Z) - Patch-Wise Self-Supervised Visual Representation Learning: A Fine-Grained Approach [4.9204263448542465]
This study introduces an innovative, fine-grained dimension by integrating patch-level discrimination into self-supervised visual representation learning.
We employ a distinctive photometric patch-level augmentation, where each patch is individually augmented, independent from other patches within the same view.
We present a simple yet effective patch-matching algorithm to find the corresponding patches across the augmented views.
arXiv Detail & Related papers (2023-10-28T09:35:30Z) - Multi-Spectral Image Stitching via Spatial Graph Reasoning [52.27796682972484]
We propose a spatial graph reasoning based multi-spectral image stitching method.
We embed multi-scale complementary features from the same view position into a set of nodes.
By introducing long-range coherence along spatial and channel dimensions, the complementarity of pixel relations and channel interdependencies aids in the reconstruction of aligned multi-view features.
arXiv Detail & Related papers (2023-07-31T15:04:52Z) - Cross-View Hierarchy Network for Stereo Image Super-Resolution [14.574538513341277]
Stereo image super-resolution aims to improve the quality of high-resolution stereo image pairs by exploiting complementary information across views.
We propose a novel method, named Cross-View-Hierarchy Network for Stereo Image Super-Resolution (CVHSSR)
CVHSSR achieves the best stereo image super-resolution performance than other state-of-the-art methods while using fewer parameters.
arXiv Detail & Related papers (2023-04-13T03:11:30Z) - ViCE: Self-Supervised Visual Concept Embeddings as Contextual and Pixel
Appearance Invariant Semantic Representations [77.3590853897664]
This work presents a self-supervised method to learn dense semantically rich visual embeddings for images inspired by methods for learning word embeddings in NLP.
arXiv Detail & Related papers (2021-11-24T12:27:30Z) - Bridging the Visual Gap: Wide-Range Image Blending [16.464837892640812]
We introduce an effective deep-learning model to realize wide-range image blending.
We experimentally demonstrate that our proposed method is able to produce visually appealing results.
arXiv Detail & Related papers (2021-03-28T15:07:45Z) - PVSNet: Pixelwise Visibility-Aware Multi-View Stereo Network [32.41293572426403]
A Pixelwise Visibility-aware multi-view Stereo Network (PVSNet) is proposed for robust dense 3D reconstruction.
PVSNet is the first deep learning framework that is able to capture the visibility information of different neighboring views.
Experiments show that PVSNet achieves the state-of-the-art performance on different datasets.
arXiv Detail & Related papers (2020-07-15T14:39:49Z) - Distilling Localization for Self-Supervised Representation Learning [82.79808902674282]
Contrastive learning has revolutionized unsupervised representation learning.
Current contrastive models are ineffective at localizing the foreground object.
We propose a data-driven approach for learning in variance to backgrounds.
arXiv Detail & Related papers (2020-04-14T16:29:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.