Textureless-aware Segmentation and Correlative Refinement Guided
Multi-View Stereo
- URL: http://arxiv.org/abs/2308.09990v2
- Date: Mon, 26 Feb 2024 17:27:22 GMT
- Title: Textureless-aware Segmentation and Correlative Refinement Guided
Multi-View Stereo
- Authors: Zhenlong Yuan, Jiakai Cao, Hao Jiang, Zhaoqi Wang and Zhaoxin Li
- Abstract summary: We present a Textureless-aware And Correlative Refinement guided Multi-View Stereo.
It tackles challenges posed by textureless areas in 3D reconstruction through filtering, refinement and segmentation.
Our method significantly outperforms most non-learning methods and exhibits robustness to textureless areas while preserving fine details.
- Score: 6.886220026399107
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The reconstruction of textureless areas has long been a challenging problem
in MVS due to lack of reliable pixel correspondences between images. In this
paper, we propose the Textureless-aware Segmentation And Correlative Refinement
guided Multi-View Stereo, a novel method that effectively tackles challenges
posed by textureless areas in 3D reconstruction through filtering, refinement
and segmentation. First, we implement joint hypothesis filtering, a technique
that merges a confidence estimator with a disparity discontinuity detector to
eliminate incorrect depth estimations. Second, to spread the pixels with
confident depth, we introduce a iterative correlation refinement strategy that
leverages RANSAC to generate superpixels, succeeded by a median filter for
broadening the influence of accurately determined pixels.Finally, we present a
textureless-aware segmentation method that leverages edge detection and line
detection for accurately identify large textureless regions to be fitted using
3D planes. Experiments on extensive datasets demonstrate that our method
significantly outperforms most non-learning methods and exhibits robustness to
textureless areas while preserving fine details.
Related papers
- Depth-aware Volume Attention for Texture-less Stereo Matching [67.46404479356896]
We propose a lightweight volume refinement scheme to tackle the texture deterioration in practical outdoor scenarios.
We introduce a depth volume supervised by the ground-truth depth map, capturing the relative hierarchy of image texture.
Local fine structure and context are emphasized to mitigate ambiguity and redundancy during volume aggregation.
arXiv Detail & Related papers (2024-02-14T04:07:44Z) - SD-MVS: Segmentation-Driven Deformation Multi-View Stereo with Spherical
Refinement and EM optimization [6.886220026399106]
We introduce Multi-View Stereo (SD-MVS) to tackle challenges in 3D reconstruction of textureless areas.
We are the first to adopt the Segment Anything Model (SAM) to distinguish semantic instances in scenes.
We propose a unique refinement strategy that combines spherical coordinates and gradient descent on normals and pixelwise search interval on depths.
arXiv Detail & Related papers (2024-01-12T05:25:57Z) - VoxelNextFusion: A Simple, Unified and Effective Voxel Fusion Framework
for Multi-Modal 3D Object Detection [33.46363259200292]
Existing voxel-based methods face challenges when fusing sparse voxel features with dense image features in a one-to-one manner.
We present VoxelNextFusion, a multi-modal 3D object detection framework specifically designed for voxel-based methods.
arXiv Detail & Related papers (2024-01-05T08:10:49Z) - RCDN -- Robust X-Corner Detection Algorithm based on Advanced CNN Model [3.580983453285039]
We present a novel detection algorithm which can maintain high sub-pixel precision on inputs under multiple interferences.
The whole algorithm, adopting a coarse-to-fine strategy, contains a X-corner detection network and three post-processing techniques.
Evaluations on real and synthetic images indicate that the presented algorithm has the higher detection rate, sub-pixel accuracy and robustness than other commonly used methods.
arXiv Detail & Related papers (2023-07-07T10:40:41Z) - Parallax-Tolerant Unsupervised Deep Image Stitching [57.76737888499145]
We propose UDIS++, a parallax-tolerant unsupervised deep image stitching technique.
First, we propose a robust and flexible warp to model the image registration from global homography to local thin-plate spline motion.
To further eliminate the parallax artifacts, we propose to composite the stitched image seamlessly by unsupervised learning for seam-driven composition masks.
arXiv Detail & Related papers (2023-02-16T10:40:55Z) - High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization [51.878078860524795]
We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views.
Our approach enables high-fidelity 3D rendering from a single image, which is promising for various applications of AI-generated 3D content.
arXiv Detail & Related papers (2022-11-28T18:59:52Z) - PAEDID: Patch Autoencoder Based Deep Image Decomposition For Pixel-level
Defective Region Segmentation [16.519583839906904]
Unsupervised patch autoencoder based deep image decomposition (PAEDID) method for defective region segmentation is presented.
We learn the common background as a deep image prior by a patch autoencoder (PAE) network.
By adopting the proposed approach, the defective regions in the image can be accurately extracted in an unsupervised fashion.
arXiv Detail & Related papers (2022-03-28T02:50:06Z) - PatchMVSNet: Patch-wise Unsupervised Multi-View Stereo for
Weakly-Textured Surface Reconstruction [2.9896482273918434]
This paper proposes robust loss functions leveraging constraints beneath multi-view images to alleviate matching ambiguity.
Our strategy can be implemented with arbitrary depth estimation frameworks and can be trained with arbitrary large-scale MVS datasets.
Our method reaches the performance of the state-of-the-art methods on popular benchmarks, like DTU, Tanks and Temples and ETH3D.
arXiv Detail & Related papers (2022-03-04T07:05:23Z) - Towards Unpaired Depth Enhancement and Super-Resolution in the Wild [121.96527719530305]
State-of-the-art data-driven methods of depth map super-resolution rely on registered pairs of low- and high-resolution depth maps of the same scenes.
We consider an approach to depth map enhancement based on learning from unpaired data.
arXiv Detail & Related papers (2021-05-25T16:19:16Z) - M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information.
In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection.
We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
arXiv Detail & Related papers (2021-04-20T05:43:44Z) - Light Field Reconstruction via Deep Adaptive Fusion of Hybrid Lenses [67.01164492518481]
This paper explores the problem of reconstructing high-resolution light field (LF) images from hybrid lenses.
We propose a novel end-to-end learning-based approach, which can comprehensively utilize the specific characteristics of the input.
Our framework could potentially decrease the cost of high-resolution LF data acquisition and benefit LF data storage and transmission.
arXiv Detail & Related papers (2021-02-14T06:44:47Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.