From One to the Power of Many: Invariance to Multi-LiDAR Perception from Single-Sensor Datasets

• URL: http://arxiv.org/abs/2409.18592v2
• Date: Fri, 24 Jan 2025 17:21:45 GMT
• Title: From One to the Power of Many: Invariance to Multi-LiDAR Perception from Single-Sensor Datasets
• Authors: Marc Uecker, J. Marius Zöllner,
• Abstract summary: We introduce a new metric for feature-level invariance which can serve as a proxy to measure cross-domain generalization without requiring labeled data. We propose two application-specific data augmentations, which facilitate better transfer to multi-sensor setups LiDAR, when trained on single-sensor datasets.
• Score: 12.712896458348515
• License:
• Abstract: Recently, LiDAR segmentation methods for autonomous vehicles, powered by deep neural networks, have experienced steep growth in performance on classic benchmarks, such as nuScenes and SemanticKITTI. However, there are still large gaps in performance when deploying models trained on such single-sensor setups to modern vehicles with multiple high-resolution LiDAR sensors. In this work, we introduce a new metric for feature-level invariance which can serve as a proxy to measure cross-domain generalization without requiring labeled data. Additionally, we propose two application-specific data augmentations, which facilitate better transfer to multi-sensor LiDAR setups, when trained on single-sensor datasets. We provide experimental evidence on both simulated and real data, that our proposed augmentations improve invariance across LiDAR setups, leading to improved generalization.

Related papers

• Finetuning Pre-trained Model with Limited Data for LiDAR-based 3D Object Detection by Bridging Domain Gaps [8.897884780881535]
  LiDAR-based 3D object detectors often fail to adapt well to target domains with different sensor configurations. Recent studies suggest that pre-trained backbones can be learned in a self-supervised manner with large-scale unlabeled LiDAR frames. We propose a novel method, called Domain Adaptive Distill-Tuning (DADT), to adapt a pre-trained model with limited target data.
  arXiv  Detail & Related papers  (2024-10-02T08:22:42Z)
• SFPNet: Sparse Focal Point Network for Semantic Segmentation on General LiDAR Point Clouds [13.097858142421519]
  We propose a framework to accommodate various types of LiDAR prevalent in the market by replacing window-attention with sparse focal point modulation. Our SFPNet is capable of extracting multi-level contexts and dynamically aggregating them using a gate mechanism. We also introduce a novel large-scale hybrid-solid LiDAR semantic segmentation dataset for robotic applications.
  arXiv  Detail & Related papers  (2024-07-16T10:22:09Z)
• Cross-sensor self-supervised training and alignment for remote sensing [2.1178416840822027]
  We introduce cross-sensor self-supervised training and alignment for remote sensing (X-STARS) X-STARS can be applied to train models from scratch, or adapt large models pretrained on e.g low-resolution data to new high-resolution sensors. We demonstrate that X-STARS outperforms the state-of-the-art by a significant margin with less data across various conditions of data availability and resolutions.
  arXiv  Detail & Related papers  (2024-05-16T09:25:45Z)
• Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving [58.16024314532443]
  We introduce LaserMix++, a framework that integrates laser beam manipulations from disparate LiDAR scans and incorporates LiDAR-camera correspondences to assist data-efficient learning. Results demonstrate that LaserMix++ outperforms fully supervised alternatives, achieving comparable accuracy with five times fewer annotations. This substantial advancement underscores the potential of semi-supervised approaches in reducing the reliance on extensive labeled data in LiDAR-based 3D scene understanding systems.
  arXiv  Detail & Related papers  (2024-05-08T17:59:53Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• Instant Domain Augmentation for LiDAR Semantic Segmentation [10.250046817380458]
  This paper presents a fast and flexible LiDAR augmentation method for the semantic segmentation task, called 'LiDomAug'. Our on-demand augmentation module runs at 330 FPS, so it can be seamlessly integrated into the data loader in the learning framework.
  arXiv  Detail & Related papers  (2023-03-25T06:59:12Z)
• Learning to Simulate Realistic LiDARs [66.7519667383175]
  We introduce a pipeline for data-driven simulation of a realistic LiDAR sensor. We show that our model can learn to encode realistic effects such as dropped points on transparent surfaces. We use our technique to learn models of two distinct LiDAR sensors and use them to improve simulated LiDAR data accordingly.
  arXiv  Detail & Related papers  (2022-09-22T13:12:54Z)
• LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection [96.63947479020631]
  In many real-world applications, the LiDAR points used by mass-produced robots and vehicles usually have fewer beams than that in large-scale public datasets. We propose the LiDAR Distillation to bridge the domain gap induced by different LiDAR beams for 3D object detection.
  arXiv  Detail & Related papers  (2022-03-28T17:59:02Z)
• Learning Moving-Object Tracking with FMCW LiDAR [53.05551269151209]
  We propose a learning-based moving-object tracking method utilizing our newly developed LiDAR sensor, Frequency Modulated Continuous Wave (FMCW) LiDAR. Given the labels, we propose a contrastive learning framework, which pulls together the features from the same instance in embedding space and pushes apart the features from different instances to improve the tracking quality.
  arXiv  Detail & Related papers  (2022-03-02T09:11:36Z)
• SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devices [69.1412199244903]
  We present a multi-tenant runtime for adaptive model execution with integrated MLOps on edge devices, e.g., a camera, a microphone, or IoT sensors. S SensiX++ operates on two fundamental principles - highly modular componentisation to externalise data operations with clear abstractions and document-centric manifestation for system-wide orchestration. We report on the overall throughput and quantified benefits of various automation components of SensiX++ and demonstrate its efficacy to significantly reduce operational complexity and lower the effort to deploy, upgrade, reconfigure and serve embedded models on edge devices.
  arXiv  Detail & Related papers  (2021-09-08T22:06:16Z)
• High-Precision Digital Traffic Recording with Multi-LiDAR Infrastructure Sensor Setups [0.0]
  We investigate the impact of fused LiDAR point clouds compared to single LiDAR point clouds. The evaluation of the extracted trajectories shows that a fused infrastructure approach significantly increases the tracking results and reaches accuracies within a few centimeters.
  arXiv  Detail & Related papers  (2020-06-22T10:57:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.