Graph-based Thermal-Inertial SLAM with Probabilistic Neural Networks

• URL: http://arxiv.org/abs/2104.07196v2
• Date: Sun, 18 Apr 2021 08:35:26 GMT
• Title: Graph-based Thermal-Inertial SLAM with Probabilistic Neural Networks
• Authors: Muhamad Risqi U. Saputra, Chris Xiaoxuan Lu, Pedro P. B. de Gusmao, Bing Wang, Andrew Markham, Niki Trigoni
• Abstract summary: We propose the first complete thermal-inertial SLAM system which combines neural abstraction in the SLAM front end with robust pose graph optimization in the SLAM back end. Our key strategies to successfully model this encoding from thermal imagery are the usage of normalized 14-bit radiometric data, the incorporation of hallucinated visual (RGB) features, and the inclusion of feature selection to estimate the MDN parameters.
• Score: 38.35547654117047
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Simultaneous Localization and Mapping (SLAM) system typically employ vision-based sensors to observe the surrounding environment. However, the performance of such systems highly depends on the ambient illumination conditions. In scenarios with adverse visibility or in the presence of airborne particulates (e.g. smoke, dust, etc.), alternative modalities such as those based on thermal imaging and inertial sensors are more promising. In this paper, we propose the first complete thermal-inertial SLAM system which combines neural abstraction in the SLAM front end with robust pose graph optimization in the SLAM back end. We model the sensor abstraction in the front end by employing probabilistic deep learning parameterized by Mixture Density Networks (MDN). Our key strategies to successfully model this encoding from thermal imagery are the usage of normalized 14-bit radiometric data, the incorporation of hallucinated visual (RGB) features, and the inclusion of feature selection to estimate the MDN parameters. To enable a full SLAM system, we also design an efficient global image descriptor which is able to detect loop closures from thermal embedding vectors. We performed extensive experiments and analysis using three datasets, namely self-collected ground robot and handheld data taken in indoor environment, and one public dataset (SubT-tunnel) collected in underground tunnel. Finally, we demonstrate that an accurate thermal-inertial SLAM system can be realized in conditions of both benign and adverse visibility.

Related papers

• Deep Generative Data Assimilation in Multimodal Setting [0.1052166918701117]
  In this work, we propose SLAMS: Score-based Latent Assimilation in Multimodal Setting. We assimilate in-situ weather station data and ex-situ satellite imagery to calibrate the vertical temperature profiles, globally. Our work is the first to apply deep generative framework for multimodal data assimilation using real-world datasets.
  arXiv  Detail & Related papers  (2024-04-10T00:25:09Z)
• Ternary-Type Opacity and Hybrid Odometry for RGB NeRF-SLAM [58.736472371951955]
  We introduce a ternary-type opacity (TT) model, which categorizes points on a ray intersecting a surface into three regions: before, on, and behind the surface. This enables a more accurate rendering of depth, subsequently improving the performance of image warping techniques. Our integrated approach of TT and HO achieves state-of-the-art performance on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-12-20T18:03:17Z)
• DNS SLAM: Dense Neural Semantic-Informed SLAM [92.39687553022605]
  DNS SLAM is a novel neural RGB-D semantic SLAM approach featuring a hybrid representation. Our method integrates multi-view geometry constraints with image-based feature extraction to improve appearance details. Our experimental results achieve state-of-the-art performance on both synthetic data and real-world data tracking.
  arXiv  Detail & Related papers  (2023-11-30T21:34:44Z)
• Spatially-resolved Thermometry from Line-of-Sight Emission Spectroscopy via Machine Learning [2.449329947677678]
  The aim of this research is to explore the use of data-driven models in measuring temperature distributions. Two categories of data-driven methods are analyzed: (i) Feature engineering and classical machine learning algorithms, and (ii) end-to-end convolutional neural networks (CNN) The proposed method is capable of measuring nonuniform temperature distributions from low-resolution spectra, even when the species concentration distribution in the gas mixtures is unknown.
  arXiv  Detail & Related papers  (2022-12-15T13:46:15Z)
• ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields [2.0625936401496237]
  ESLAM reads RGB-D frames with unknown camera poses in a sequential manner and incrementally reconstructs the scene representation. ESLAM improves the accuracy of 3D reconstruction and camera localization of state-of-the-art dense visual SLAM methods by more than 50%.
  arXiv  Detail & Related papers  (2022-11-21T18:25:14Z)
• Environmental Sensor Placement with Convolutional Gaussian Neural Processes [65.13973319334625]
  It is challenging to place sensors in a way that maximises the informativeness of their measurements, particularly in remote regions like Antarctica. Probabilistic machine learning models can suggest informative sensor placements by finding sites that maximally reduce prediction uncertainty. This paper proposes using a convolutional Gaussian neural process (ConvGNP) to address these issues.
  arXiv  Detail & Related papers  (2022-11-18T17:25:14Z)
• Using Detection, Tracking and Prediction in Visual SLAM to Achieve Real-time Semantic Mapping of Dynamic Scenarios [70.70421502784598]
  RDS-SLAM can build semantic maps at object level for dynamic scenarios in real time using only one commonly used Intel Core i7 CPU. We evaluate RDS-SLAM in TUM RGB-D dataset, and experimental results show that RDS-SLAM can run with 30.3 ms per frame in dynamic scenarios.
  arXiv  Detail & Related papers  (2022-10-10T11:03:32Z)
• Robust Environment Perception for Automated Driving: A Unified Learning Pipeline for Visual-Infrared Object Detection [2.478658210785]
  We exploit both visual and thermal perception units for robust object detection purposes. In this paper, we exploit both visual and thermal perception units for robust object detection purposes.
  arXiv  Detail & Related papers  (2022-06-08T15:02:58Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• SSTN: Self-Supervised Domain Adaptation Thermal Object Detection for Autonomous Driving [6.810856082577402]
  We have proposed a deep neural network Self Supervised Thermal Network (SSTN) for learning the feature embedding to maximize the information between visible and infrared spectrum domain by contrastive learning. The proposed method is extensively evaluated on the two publicly available datasets: the FLIR-ADAS dataset and the KAIST Multi-Spectral dataset.
  arXiv  Detail & Related papers  (2021-03-04T16:42:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.