Sensor Equivariance by LiDAR Projection Images

• URL: http://arxiv.org/abs/2305.00221v1
• Date: Sat, 29 Apr 2023 10:16:02 GMT
• Title: Sensor Equivariance by LiDAR Projection Images
• Authors: Hannes Reichert, Manuel Hetzel, Steven Schreck, Konrad Doll, and Bernhard Sick
• Abstract summary: We propose an extension of conventional image data by an additional channel in which the associated projection properties are encoded. This addresses the issue of sensor-dependent object representation in projection-based sensors, such as LiDAR.
• Score: 4.413278371057897
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In this work, we propose an extension of conventional image data by an additional channel in which the associated projection properties are encoded. This addresses the issue of sensor-dependent object representation in projection-based sensors, such as LiDAR, which can lead to distorted physical and geometric properties due to variations in sensor resolution and field of view. To that end, we propose an architecture for processing this data in an instance segmentation framework. We focus specifically on LiDAR as a key sensor modality for machine vision tasks and highly automated driving (HAD). Through an experimental setup in a controlled synthetic environment, we identify a bias on sensor resolution and field of view and demonstrate that our proposed method can reduce said bias for the task of LiDAR instance segmentation. Furthermore, we define our method such that it can be applied to other projection-based sensors, such as cameras. To promote transparency, we make our code and dataset publicly available. This method shows the potential to improve performance and robustness in various machine vision tasks that utilize projection-based sensors.

Related papers

• Descriptor: Face Detection Dataset for Programmable Threshold-Based Sparse-Vision [0.8271394038014485]
  This dataset is an annotated, temporal-threshold-based vision dataset for face detection tasks derived from the same videos used for Aff-Wild2. We anticipate that this resource will significantly support the development of robust vision systems based on smart sensors that can process based on temporal-difference thresholds.
  arXiv  Detail & Related papers  (2024-10-01T03:42:03Z)
• SenPa-MAE: Sensor Parameter Aware Masked Autoencoder for Multi-Satellite Self-Supervised Pretraining [1.4528189330418977]
  SenPa-MAE encodes the sensor parameters of an observed multispectral signal into the image embeddings. SenPa-MAE can be pre-trained on imagery of different satellites with non-matching spectral or geometrical sensor characteristics.
  arXiv  Detail & Related papers  (2024-08-20T16:53:30Z)
• Semantic Guided Large Scale Factor Remote Sensing Image Super-resolution with Generative Diffusion Prior [13.148815217684277]
  Large scale factor super-resolution (SR) algorithms are vital for maximizing the utilization of low-resolution (LR) satellite data captured from orbit. Existing methods confront challenges in recovering SR images with clear textures and correct ground objects. We introduce a novel framework, the Semantic Guided Diffusion Model (SGDM), designed for large scale factor remote sensing image super-resolution.
  arXiv  Detail & Related papers  (2024-05-11T16:06:16Z)
• Learning to Simulate Realistic LiDARs [66.7519667383175]
  We introduce a pipeline for data-driven simulation of a realistic LiDAR sensor. We show that our model can learn to encode realistic effects such as dropped points on transparent surfaces. We use our technique to learn models of two distinct LiDAR sensors and use them to improve simulated LiDAR data accordingly.
  arXiv  Detail & Related papers  (2022-09-22T13:12:54Z)
• Multitask AET with Orthogonal Tangent Regularity for Dark Object Detection [84.52197307286681]
  We propose a novel multitask auto encoding transformation (MAET) model to enhance object detection in a dark environment. In a self-supervision manner, the MAET learns the intrinsic visual structure by encoding and decoding the realistic illumination-degrading transformation. We have achieved the state-of-the-art performance using synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-05-06T16:27:14Z)
• Mapping LiDAR and Camera Measurements in a Dual Top-View Grid Representation Tailored for Automated Vehicles [3.337790639927531]
  We present a generic evidential grid mapping pipeline designed for imaging sensors such as LiDARs and cameras. Our grid-based evidential model contains semantic estimates for cell occupancy and ground separately. Our method estimates cell occupancy robustly and with a high level of detail while maximizing efficiency and minimizing the dependency to external processing modules.
  arXiv  Detail & Related papers  (2022-04-16T23:51:20Z)
• SSTN: Self-Supervised Domain Adaptation Thermal Object Detection for Autonomous Driving [6.810856082577402]
  We have proposed a deep neural network Self Supervised Thermal Network (SSTN) for learning the feature embedding to maximize the information between visible and infrared spectrum domain by contrastive learning. The proposed method is extensively evaluated on the two publicly available datasets: the FLIR-ADAS dataset and the KAIST Multi-Spectral dataset.
  arXiv  Detail & Related papers  (2021-03-04T16:42:49Z)
• Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups [68.8204255655161]
  We present a method to calibrate the parameters of any pair of sensors involving LiDARs, monocular or stereo cameras. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups.
  arXiv  Detail & Related papers  (2021-01-12T12:02:26Z)
• Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition [131.6328804788164]
  We propose a framework, named Semantics-aware Adaptive Knowledge Distillation Networks (SAKDN), to enhance action recognition in vision-sensor modality (videos) The SAKDN uses multiple wearable-sensors as teacher modalities and uses RGB videos as student modality.
  arXiv  Detail & Related papers  (2020-09-01T03:38:31Z)
• Learning Camera Miscalibration Detection [83.38916296044394]
  This paper focuses on a data-driven approach to learn the detection of miscalibration in vision sensors, specifically RGB cameras. Our contributions include a proposed miscalibration metric for RGB cameras and a novel semi-synthetic dataset generation pipeline based on this metric. By training a deep convolutional neural network, we demonstrate the effectiveness of our pipeline to identify whether a recalibration of the camera's intrinsic parameters is required or not.
  arXiv  Detail & Related papers  (2020-05-24T10:32:49Z)
• Deep Soft Procrustes for Markerless Volumetric Sensor Alignment [81.13055566952221]
  In this work, we improve markerless data-driven correspondence estimation to achieve more robust multi-sensor spatial alignment. We incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure.
  arXiv  Detail & Related papers  (2020-03-23T10:51:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.