Attention-Guided Lidar Segmentation and Odometry Using Image-to-Point Cloud Saliency Transfer
- URL: http://arxiv.org/abs/2308.14332v2
- Date: Mon, 17 Jun 2024 00:44:02 GMT
- Title: Attention-Guided Lidar Segmentation and Odometry Using Image-to-Point Cloud Saliency Transfer
- Authors: Guanqun Ding, Nevrez Imamoglu, Ali Caglayan, Masahiro Murakawa, Ryosuke Nakamura,
- Abstract summary: SalLiDAR is a saliency-guided 3D semantic segmentation model that integrates saliency information to improve segmentation performance.
SalLONet is a self-supervised saliency-guided LiDAR odometry network that uses the semantic and saliency predictions of SalLiDAR to achieve better odometry estimation.
- Score: 6.058427379240697
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: LiDAR odometry estimation and 3D semantic segmentation are crucial for autonomous driving, which has achieved remarkable advances recently. However, these tasks are challenging due to the imbalance of points in different semantic categories for 3D semantic segmentation and the influence of dynamic objects for LiDAR odometry estimation, which increases the importance of using representative/salient landmarks as reference points for robust feature learning. To address these challenges, we propose a saliency-guided approach that leverages attention information to improve the performance of LiDAR odometry estimation and semantic segmentation models. Unlike in the image domain, only a few studies have addressed point cloud saliency information due to the lack of annotated training data. To alleviate this, we first present a universal framework to transfer saliency distribution knowledge from color images to point clouds, and use this to construct a pseudo-saliency dataset (i.e. FordSaliency) for point clouds. Then, we adopt point cloud-based backbones to learn saliency distribution from pseudo-saliency labels, which is followed by our proposed SalLiDAR module. SalLiDAR is a saliency-guided 3D semantic segmentation model that integrates saliency information to improve segmentation performance. Finally, we introduce SalLONet, a self-supervised saliency-guided LiDAR odometry network that uses the semantic and saliency predictions of SalLiDAR to achieve better odometry estimation. Our extensive experiments on benchmark datasets demonstrate that the proposed SalLiDAR and SalLONet models achieve state-of-the-art performance against existing methods, highlighting the effectiveness of image-to-LiDAR saliency knowledge transfer. Source code will be available at https://github.com/nevrez/SalLONet.
Related papers
- 4D Contrastive Superflows are Dense 3D Representation Learners [62.433137130087445]
We introduce SuperFlow, a novel framework designed to harness consecutive LiDAR-camera pairs for establishing pretraining objectives.
To further boost learning efficiency, we incorporate a plug-and-play view consistency module that enhances alignment of the knowledge distilled from camera views.
arXiv Detail & Related papers (2024-07-08T17:59:54Z) - Off-Road LiDAR Intensity Based Semantic Segmentation [11.684330305297523]
Learning-based LiDAR semantic segmentation utilizes machine learning techniques to automatically classify objects in LiDAR point clouds.
We address this problem by harnessing the LiDAR intensity parameter to enhance object segmentation in off-road environments.
Our approach was evaluated in the RELLIS-3D data set and yielded promising results as a preliminary analysis with improved mIoU for classes "puddle" and "grass"
arXiv Detail & Related papers (2024-01-02T21:27:43Z) - Refining the ONCE Benchmark with Hyperparameter Tuning [45.55545585587993]
This work focuses on the evaluation of semi-supervised learning approaches for point cloud data.
Data annotation is of paramount importance in the context of LiDAR applications.
We show that improvements from previous semi-supervised methods may not be as profound as previously thought.
arXiv Detail & Related papers (2023-11-10T13:39:07Z) - NeRF-LOAM: Neural Implicit Representation for Large-Scale Incremental
LiDAR Odometry and Mapping [14.433784957457632]
We propose a novel NeRF-based LiDAR odometry and mapping approach, NeRF-LOAM, consisting of three modules neural odometry, neural mapping, and mesh reconstruction.
Our approach achieves state-of-the-art odometry and mapping performance, as well as a strong generalization in large-scale environments utilizing LiDAR data.
arXiv Detail & Related papers (2023-03-19T16:40:36Z) - Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding [62.17020485045456]
It is commonly assumed in semi-supervised learning (SSL) that the unlabeled data are drawn from the same distribution as that of the labeled ones.
We propose to selectively utilize unlabeled data through sample weighting, so that only conducive unlabeled data would be prioritized.
arXiv Detail & Related papers (2022-05-02T16:09:17Z) - Learning Where to Learn in Cross-View Self-Supervised Learning [54.14989750044489]
Self-supervised learning (SSL) has made enormous progress and largely narrowed the gap with supervised ones.
Current methods simply adopt uniform aggregation of pixels for embedding.
We present a new approach, Learning Where to Learn (LEWEL), to adaptively aggregate spatial information of features.
arXiv Detail & Related papers (2022-03-28T17:02:42Z) - Guided Point Contrastive Learning for Semi-supervised Point Cloud
Semantic Segmentation [90.2445084743881]
We present a method for semi-supervised point cloud semantic segmentation to adopt unlabeled point clouds in training to boost the model performance.
Inspired by the recent contrastive loss in self-supervised tasks, we propose the guided point contrastive loss to enhance the feature representation and model generalization ability.
arXiv Detail & Related papers (2021-10-15T16:38:54Z) - SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural
Networks [81.64530401885476]
We propose a self-supervised LiDAR odometry method, dubbed SelfVoxeLO, to tackle these two difficulties.
Specifically, we propose a 3D convolution network to process the raw LiDAR data directly, which extracts features that better encode the 3D geometric patterns.
We evaluate our method's performances on two large-scale datasets, i.e., KITTI and Apollo-SouthBay.
arXiv Detail & Related papers (2020-10-19T09:23:39Z) - LodoNet: A Deep Neural Network with 2D Keypoint Matchingfor 3D LiDAR
Odometry Estimation [22.664095688406412]
We propose to transfer the LiDAR frames to image space and reformulate the problem as image feature extraction.
With the help of scale-invariant feature transform (SIFT) for feature extraction, we are able to generate matched keypoint pairs (MKPs)
A convolutional neural network pipeline is designed for LiDAR odometry estimation by extracted MKPs.
The proposed scheme, namely LodoNet, is then evaluated in the KITTI odometry estimation benchmark, achieving on par with or even better results than the state-of-the-art.
arXiv Detail & Related papers (2020-09-01T01:09:41Z) - Complete & Label: A Domain Adaptation Approach to Semantic Segmentation
of LiDAR Point Clouds [49.47017280475232]
We study an unsupervised domain adaptation problem for the semantic labeling of 3D point clouds.
We take a Complete and Label approach to recover the underlying surfaces before passing them to a segmentation network.
The recovered 3D surfaces serve as a canonical domain, from which semantic labels can transfer across different LiDAR sensors.
arXiv Detail & Related papers (2020-07-16T17:42:05Z) - Scan-based Semantic Segmentation of LiDAR Point Clouds: An Experimental
Study [2.6205925938720833]
State of the art methods use deep neural networks to predict semantic classes for each point in a LiDAR scan.
A powerful and efficient way to process LiDAR measurements is to use two-dimensional, image-like projections.
We demonstrate various techniques to boost the performance and to improve runtime as well as memory constraints.
arXiv Detail & Related papers (2020-04-06T11:08:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.