Integrating Multi-Modal Sensors: A Review of Fusion Techniques for Intelligent Vehicles

• URL: http://arxiv.org/abs/2506.21885v1
• Date: Fri, 27 Jun 2025 03:43:48 GMT
• Title: Integrating Multi-Modal Sensors: A Review of Fusion Techniques for Intelligent Vehicles
• Authors: Chuheng Wei, Ziye Qin, Ziyan Zhang, Guoyuan Wu, Matthew J. Barth,
• Abstract summary: Multi-sensor fusion plays a critical role in enhancing perception for autonomous driving.<n>This paper formalizes multi-sensor fusion strategies into data-level, feature-level, and decision-level categories.<n>We present key multi-modal datasets and discuss their applicability in addressing real-world challenges.
• Score: 11.412978676426205
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-sensor fusion plays a critical role in enhancing perception for autonomous driving, overcoming individual sensor limitations, and enabling comprehensive environmental understanding. This paper first formalizes multi-sensor fusion strategies into data-level, feature-level, and decision-level categories and then provides a systematic review of deep learning-based methods corresponding to each strategy. We present key multi-modal datasets and discuss their applicability in addressing real-world challenges, particularly in adverse weather conditions and complex urban environments. Additionally, we explore emerging trends, including the integration of Vision-Language Models (VLMs), Large Language Models (LLMs), and the role of sensor fusion in end-to-end autonomous driving, highlighting its potential to enhance system adaptability and robustness. Our work offers valuable insights into current methods and future directions for multi-sensor fusion in autonomous driving.

Related papers

• Generative AI for Autonomous Driving: Frontiers and Opportunities [145.6465312554513]
  This survey delivers a comprehensive synthesis of the emerging role of GenAI across the autonomous driving stack.<n>We begin by distilling the principles and trade-offs of modern generative modeling, encompassing VAEs, GANs, Diffusion Models, and Large Language Models.<n>We categorize practical applications, such as synthetic data generalization, end-to-end driving strategies, high-fidelity digital twin systems, smart transportation networks, and cross-domain transfer to embodied AI.
  arXiv  Detail & Related papers  (2025-05-13T17:59:20Z)
• VALISENS: A Validated Innovative Multi-Sensor System for Cooperative Automated Driving [0.9527960631238174]
  This paper presents VALISENS, an innovative multi-sensor system distributed across multiple agents.<n>It integrates onboard and roadside LiDARs, radars, thermal cameras, and RGB cameras to enhance situational awareness and support cooperative automated driving.<n>The proposed system demonstrates the potential of cooperative perception in real-world test environments.
  arXiv  Detail & Related papers  (2025-05-11T13:41:37Z)
• A Survey of World Models for Autonomous Driving [63.33363128964687]
  Recent breakthroughs in autonomous driving have been propelled by advances in robust world modeling.<n>World models offer high-fidelity representations of the driving environment that integrate multi-sensor data, semantic cues, and temporal dynamics.<n>This paper systematically reviews recent advances in world models for autonomous driving.
  arXiv  Detail & Related papers  (2025-01-20T04:00:02Z)
• Graph-Based Multi-Modal Sensor Fusion for Autonomous Driving [3.770103075126785]
  We introduce a novel approach to multi-modal sensor fusion, focusing on developing a graph-based state representation. We present a Sensor-Agnostic Graph-Aware Kalman Filter, the first online state estimation technique designed to fuse multi-modal graphs. We validate the effectiveness of our proposed framework through extensive experiments conducted on both synthetic and real-world driving datasets.
  arXiv  Detail & Related papers  (2024-11-06T06:58:17Z)
• Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Survey [61.39993881402787]
  World models and video generation are pivotal technologies in the domain of autonomous driving. This paper investigates the relationship between these two technologies. By analyzing the interplay between video generation and world models, this survey identifies critical challenges and future research directions.
  arXiv  Detail & Related papers  (2024-11-05T08:58:35Z)
• Integration of Mixture of Experts and Multimodal Generative AI in Internet of Vehicles: A Survey [82.84057882105931]
  Generative AI (GAI) can enhance the cognitive, reasoning, and planning capabilities of intelligent modules in the Internet of Vehicles (IoV) We present the fundamentals of GAI, MoE, and their interplay applications in IoV. We discuss the potential integration of MoE and GAI in IoV, including distributed perception and monitoring, collaborative decision-making and planning, and generative modeling and simulation.
  arXiv  Detail & Related papers  (2024-04-25T06:22:21Z)
• Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models [114.69732301904419]
  We present an approach to apply end-to-end open-set (any environment/scene) autonomous driving that is capable of providing driving decisions from representations queryable by image and text. Our approach demonstrates unparalleled results in diverse tests while achieving significantly greater robustness in out-of-distribution situations.
  arXiv  Detail & Related papers  (2023-10-26T17:56:35Z)
• Penalty-Based Imitation Learning With Cross Semantics Generation Sensor Fusion for Autonomous Driving [1.2749527861829049]
  In this paper, we provide a penalty-based imitation learning approach to integrate multiple modalities of information. We observe a remarkable increase in the driving score by more than 12% when compared to the state-of-the-art (SOTA) model, InterFuser. Our model achieves this performance enhancement while achieving a 7-fold increase in inference speed and reducing the model size by approximately 30%.
  arXiv  Detail & Related papers  (2023-03-21T14:29:52Z)
• Multi-modal Sensor Fusion for Auto Driving Perception: A Survey [29.804411344922382]
  We provide a literature review of the existing multi-modal-based methods for perception tasks in autonomous driving.<n>We propose an innovative way that divides them into two major classes, four minor classes by a more reasonable taxonomy in the view of the fusion stage.
  arXiv  Detail & Related papers  (2022-02-06T04:18:45Z)
• Learning Selective Sensor Fusion for States Estimation [47.76590539558037]
  We propose SelectFusion, an end-to-end selective sensor fusion module. During prediction, the network is able to assess the reliability of the latent features from different sensor modalities. We extensively evaluate all fusion strategies in both public datasets and on progressively degraded datasets.
  arXiv  Detail & Related papers  (2019-12-30T20:25:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.