Descriptor-Free Multi-View Region Matching for Instance-Wise 3D Reconstruction

• URL: http://arxiv.org/abs/2011.13649v1
• Date: Fri, 27 Nov 2020 10:45:18 GMT
• Title: Descriptor-Free Multi-View Region Matching for Instance-Wise 3D Reconstruction
• Authors: Takuma Doi, Fumio Okura, Toshiki Nagahara, Yasuyuki Matsushita, Yasushi Yagi
• Abstract summary: We propose a multi-view region matching method based on epipolar geometry. We show that the epipolar region matching can be easily integrated into instance segmentation and effective for instance-wise 3D reconstruction.
• Score: 34.21773285521006
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper proposes a multi-view extension of instance segmentation without relying on texture or shape descriptor matching. Multi-view instance segmentation becomes challenging for scenes with repetitive textures and shapes, e.g., plant leaves, due to the difficulty of multi-view matching using texture or shape descriptors. To this end, we propose a multi-view region matching method based on epipolar geometry, which does not rely on any feature descriptors. We further show that the epipolar region matching can be easily integrated into instance segmentation and effective for instance-wise 3D reconstruction. Experiments demonstrate the improved accuracy of multi-view instance matching and the 3D reconstruction compared to the baseline methods.

Related papers

• Hierarchical Image-Guided 3D Point Cloud Segmentation in Industrial Scenes via Multi-View Bayesian Fusion [4.679314646805623]
  3D segmentation is critical for understanding complex scenes with dense layouts and multi-scale objects.<n>Existing 3D point-based methods require costly annotations, while image-guided methods often suffer from semantic inconsistencies across views.<n>We propose a hierarchical image-guided 3D segmentation framework that progressively refines segmentation from instance-level to part-level.
  arXiv  Detail & Related papers  (2025-12-07T15:15:52Z)
• SegMASt3R: Geometry Grounded Segment Matching [23.257530861472656]
  We leverage the spatial understanding of 3D foundation models to tackle wide-baseline segment matching.<n>We propose an architecture that uses the inductive bias of these 3D foundation models to match segments across image pairs with up to 180 degree view-point change rotation.
  arXiv  Detail & Related papers  (2025-10-06T17:31:32Z)
• ViewSRD: 3D Visual Grounding via Structured Multi-View Decomposition [34.39212457455039]
  3D visual grounding aims to identify and localize objects in a 3D space based on textual descriptions.<n>We propose ViewSRD, a framework that formulates 3D visual grounding as a structured multi-view decomposition process.<n> Experiments on 3D visual grounding datasets show that ViewSRD significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-07-15T12:35:01Z)
• PanSt3R: Multi-view Consistent Panoptic Segmentation [10.781185925397493]
  We argue that relying on 2D panoptic segmentation for a problem inherently 3D and multi-view is likely suboptimal.<n>We propose a unified and integrated approach PanSt3R, which eliminates the need for test-time optimization.<n>PanSt3R is conceptually simple, yet fast and scalable, and achieves state-of-the-art performance on several benchmarks.
  arXiv  Detail & Related papers  (2025-06-26T15:02:00Z)
• PanopticRecon: Leverage Open-vocabulary Instance Segmentation for Zero-shot Panoptic Reconstruction [23.798691661418253]
  We propose a novel zero-shot panoptic reconstruction method from RGB-D images of scenes. We tackle both challenges by propagating partial labels with the aid of dense generalized features. Our method outperforms state-of-the-art methods on the indoor dataset ScanNet V2 and the outdoor dataset KITTI-360.
  arXiv  Detail & Related papers  (2024-07-01T15:06:04Z)
• View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fields [52.08335264414515]
  We learn a novel feature field within a Neural Radiance Field (NeRF) representing a 3D scene. Our method takes view-inconsistent multi-granularity 2D segmentations as input and produces a hierarchy of 3D-consistent segmentations as output. We evaluate our method and several baselines on synthetic datasets with multi-view images and multi-granular segmentation, showcasing improved accuracy and viewpoint-consistency.
  arXiv  Detail & Related papers  (2024-05-30T04:14:58Z)
• Part123: Part-aware 3D Reconstruction from a Single-view Image [54.589723979757515]
  Part123 is a novel framework for part-aware 3D reconstruction from a single-view image. We introduce contrastive learning into a neural rendering framework to learn a part-aware feature space. A clustering-based algorithm is also developed to automatically derive 3D part segmentation results from the reconstructed models.
  arXiv  Detail & Related papers  (2024-05-27T07:10:21Z)
• SD-MVS: Segmentation-Driven Deformation Multi-View Stereo with Spherical Refinement and EM optimization [6.886220026399106]
  We introduce Multi-View Stereo (SD-MVS) to tackle challenges in 3D reconstruction of textureless areas. We are the first to adopt the Segment Anything Model (SAM) to distinguish semantic instances in scenes. We propose a unique refinement strategy that combines spherical coordinates and gradient descent on normals and pixelwise search interval on depths.
  arXiv  Detail & Related papers  (2024-01-12T05:25:57Z)
• EipFormer: Emphasizing Instance Positions in 3D Instance Segmentation [51.996943482875366]
  We present a novel Transformer-based architecture, EipFormer, which comprises progressive aggregation and dual position embedding. EipFormer achieves superior or comparable performance compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2023-12-09T16:08:47Z)
• 3D Face Reconstruction with the Geometric Guidance of Facial Part Segmentation [28.104227855986185]
  3D Morphable Models (3DMMs) provide promising 3D face reconstructions in various applications. Existing methods struggle to reconstruct faces with extreme expressions due to deficiencies in supervisory signals. Part Re-projection Distance Loss (PRDL) transforms facial part segmentation into 2D points and re-projects the reconstruction onto the image plane.
  arXiv  Detail & Related papers  (2023-12-01T03:05:21Z)
• Instance-aware 3D Semantic Segmentation powered by Shape Generators and Classifiers [28.817905887080293]
  We propose a novel instance-aware approach for 3D semantic segmentation. Our method combines several geometry processing tasks supervised at instance-level to promote the consistency of the learned feature representation.
  arXiv  Detail & Related papers  (2023-11-21T02:14:16Z)
• ONeRF: Unsupervised 3D Object Segmentation from Multiple Views [59.445957699136564]
  ONeRF is a method that automatically segments and reconstructs object instances in 3D from multi-view RGB images without any additional manual annotations. The segmented 3D objects are represented using separate Neural Radiance Fields (NeRFs) which allow for various 3D scene editing and novel view rendering.
  arXiv  Detail & Related papers  (2022-11-22T06:19:37Z)
• SEG-MAT: 3D Shape Segmentation Using Medial Axis Transform [49.51977253452456]
  We present an efficient method for 3D shape segmentation based on the medial axis transform (MAT) of the input shape. Specifically, with the rich geometrical and structural information encoded in the MAT, we are able to identify the various types of junctions between different parts of a 3D shape. Our method outperforms the state-of-the-art methods in terms of segmentation quality and is also one order of magnitude faster.
  arXiv  Detail & Related papers  (2020-10-22T07:15:23Z)
• End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds [67.27510166559563]
  We propose an end-to-end framework to learn local multi-view descriptors for 3D point clouds. Our method outperforms existing local descriptors both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-03-12T15:41:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.