CMMLoc: Advancing Text-to-PointCloud Localization with Cauchy-Mixture-Model Based Framework

• URL: http://arxiv.org/abs/2503.02593v3
• Date: Thu, 20 Mar 2025 00:06:14 GMT
• Title: CMMLoc: Advancing Text-to-PointCloud Localization with Cauchy-Mixture-Model Based Framework
• Authors: Yanlong Xu, Haoxuan Qu, Jun Liu, Wenxiao Zhang, Xun Yang,
• Abstract summary: The goal of point cloud localization is to identify a 3D position using textual description in large urban environments.<n>We propose $textbfCMMLoc, an uncertainty-aware $textbfC$auchy-$textbfM$ixture-$textbfM$odel.<n>CMMLoc outperforms existing methods, achieving state-of-the-art results on the KITTI360Pose dataset.
• Score: 16.15099680732008
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The goal of point cloud localization based on linguistic description is to identify a 3D position using textual description in large urban environments, which has potential applications in various fields, such as determining the location for vehicle pickup or goods delivery. Ideally, for a textual description and its corresponding 3D location, the objects around the 3D location should be fully described in the text description. However, in practical scenarios, e.g., vehicle pickup, passengers usually describe only the part of the most significant and nearby surroundings instead of the entire environment. In response to this $\textbf{partially relevant}$ challenge, we propose $\textbf{CMMLoc}$, an uncertainty-aware $\textbf{C}$auchy-$\textbf{M}$ixture-$\textbf{M}$odel ($\textbf{CMM}$) based framework for text-to-point-cloud $\textbf{Loc}$alization. To model the uncertain semantic relations between text and point cloud, we integrate CMM constraints as a prior during the interaction between the two modalities. We further design a spatial consolidation scheme to enable adaptive aggregation of different 3D objects with varying receptive fields. To achieve precise localization, we propose a cardinal direction integration module alongside a modality pre-alignment strategy, helping capture the spatial relationships among objects and bringing the 3D objects closer to the text modality. Comprehensive experiments validate that CMMLoc outperforms existing methods, achieving state-of-the-art results on the KITTI360Pose dataset. Codes are available in this GitHub repository https://github.com/kevin301342/CMMLoc.

Related papers

• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)
• See It All: Contextualized Late Aggregation for 3D Dense Captioning [38.14179122810755]
  3D dense captioning is a task to localize objects in a 3D scene and generate descriptive sentences for each object. Recent approaches in 3D dense captioning have adopted transformer encoder-decoder frameworks from object detection to build an end-to-end pipeline without hand-crafted components. We introduce SIA (See-It-All), a transformer pipeline that engages in 3D dense captioning with a novel paradigm called late aggregation.
  arXiv  Detail & Related papers  (2024-08-14T16:19:18Z)
• Coupled Laplacian Eigenmaps for Locally-Aware 3D Rigid Point Cloud Matching [0.0]
  We propose a new technique, based on graph Laplacian eigenmaps, to match point clouds by taking into account fine local structures. To deal with the order and sign ambiguity of Laplacian eigenmaps, we introduce a new operator, called Coupled Laplacian. We show that the similarity between those aligned high-dimensional spaces provides a locally meaningful score to match shapes.
  arXiv  Detail & Related papers  (2024-02-27T10:10:12Z)
• Text2Loc: 3D Point Cloud Localization from Natural Language [49.01851743372889]
  We tackle the problem of 3D point cloud localization based on a few natural linguistic descriptions. We introduce a novel neural network, Text2Loc, that fully interprets the semantic relationship between points and text. Text2Loc improves the localization accuracy by up to $2times$ over the state-of-the-art on the KITTI360Pose dataset.
  arXiv  Detail & Related papers  (2023-11-27T16:23:01Z)
• CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detection [78.0010542552784]
  CoDet is a novel approach to learn object-level vision-language representations for open-vocabulary object detection. By grouping images that mention a shared concept in their captions, objects corresponding to the shared concept shall exhibit high co-occurrence. CoDet has superior performances and compelling scalability in open-vocabulary detection.
  arXiv  Detail & Related papers  (2023-10-25T14:31:02Z)
• 3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding [58.924180772480504]
  3D visual grounding aims to localize the target object in a 3D point cloud by a free-form language description. We propose a relation-aware one-stage framework, named 3D Relative Position-aware Network (3-Net)
  arXiv  Detail & Related papers  (2023-07-25T09:33:25Z)
• CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds [55.44204039410225]
  We present a novel two-stage fully sparse convolutional 3D object detection framework, named CAGroup3D. Our proposed method first generates some high-quality 3D proposals by leveraging the class-aware local group strategy on the object surface voxels. To recover the features of missed voxels due to incorrect voxel-wise segmentation, we build a fully sparse convolutional RoI pooling module.
  arXiv  Detail & Related papers  (2022-10-09T13:38:48Z)
• Contextual Modeling for 3D Dense Captioning on Point Clouds [85.68339840274857]
  3D dense captioning, as an emerging vision-language task, aims to identify and locate each object from a set of point clouds. We propose two separate modules, namely the Global Context Modeling (GCM) and Local Context Modeling (LCM), in a coarse-to-fine manner. Our proposed model can effectively characterize the object representations and contextual information.
  arXiv  Detail & Related papers  (2022-10-08T05:33:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.