Knowledge Graphs as World Models for Semantic Material-Aware Obstacle Handling in Autonomous Vehicles

• URL: http://arxiv.org/abs/2503.21232v1
• Date: Thu, 27 Mar 2025 07:46:45 GMT
• Title: Knowledge Graphs as World Models for Semantic Material-Aware Obstacle Handling in Autonomous Vehicles
• Authors: Ayush Bheemaiah, Seungyong Yang,
• Abstract summary: This study suggests combining sensors with a knowledge graph (KG)-based world model to improve AVs' comprehension of physical material qualities.<n>Using the CARLA autonomous driving simulator, we evaluated AV performance with and without KG integration.<n>The KG-based method improves obstacle management, which allows AVs to use material qualities to make better decisions about when to change lanes or apply emergency braking.
• Score: 0.03946288852327085
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The inability of autonomous vehicles (AVs) to infer the material properties of obstacles limits their decision-making capacity. While AVs rely on sensor systems such as cameras, LiDAR, and radar to detect obstacles, this study suggests combining sensors with a knowledge graph (KG)-based world model to improve AVs' comprehension of physical material qualities. Beyond sensor data, AVs can infer qualities such as malleability, density, and elasticity using a semantic KG that depicts the relationships between obstacles and their attributes. Using the CARLA autonomous driving simulator, we evaluated AV performance with and without KG integration. The findings demonstrate that the KG-based method improves obstacle management, which allows AVs to use material qualities to make better decisions about when to change lanes or apply emergency braking. For example, the KG-integrated AV changed lanes for hard impediments like traffic cones and successfully avoided collisions with flexible items such as plastic bags by passing over them. Compared to the control system, the KG framework demonstrated improved responsiveness to obstacles by resolving conflicting sensor data, causing emergency stops for 13.3% more cases. In addition, our method exhibits a 6.6% higher success rate in lane-changing maneuvers in experimental scenarios, particularly for larger, high-impact obstacles. While we focus particularly on autonomous driving, our work demonstrates the potential of KG-based world models to improve decision-making in embodied AI systems and scale to other domains, including robotics, healthcare, and environmental simulation.

Related papers

• Enhancing Traffic Sign Recognition On The Performance Based On Yolov8 [0.0]
  This thesis presents an enhanced YOLOv8-based detection system that integrates advanced data augmentation techniques. Experiments conducted on datasets including GTSRB, TT100K, and GTSDB demonstrate marked improvements in detection accuracy, robustness under adverse conditions, and real-time inference on edge devices.
  arXiv  Detail & Related papers  (2025-04-02T07:28:05Z)
• MSC-Bench: Benchmarking and Analyzing Multi-Sensor Corruption for Driving Perception [9.575044300747061]
  Multi-sensor fusion models play a crucial role in autonomous driving perception, particularly in tasks like 3D object detection and HD map construction.<n>These models provide essential and comprehensive static environmental information for autonomous driving systems.<n>While camera-LiDAR fusion methods have shown promising results, they often depend on complete sensor inputs.<n>This reliance can lead to low robustness and potential failures when sensors are corrupted or missing, raising significant safety concerns.<n>To tackle this challenge, we introduce the Multi-Sensor Corruption Benchmark (MSC-Bench), the first comprehensive benchmark aimed at evaluating the robustness of multi-sensor autonomous driving perception models against various sensor corruption
  arXiv  Detail & Related papers  (2025-01-02T03:38:46Z)
• XAI-based Feature Ensemble for Enhanced Anomaly Detection in Autonomous Driving Systems [1.3022753212679383]
  This paper proposes a novel feature ensemble framework that integrates multiple Explainable AI (XAI) methods. By fusing top features identified by these XAI methods across six diverse AI models, the framework creates a robust and comprehensive set of features critical for detecting anomalies. Our technique demonstrates improved accuracy, robustness, and transparency of AI models, contributing to safer and more trustworthy autonomous driving systems.
  arXiv  Detail & Related papers  (2024-10-20T14:34:48Z)
• Towards Automated Driving Violation Cause Analysis in Scenario-Based Testing for Autonomous Driving Systems [22.872694649245044]
  We propose a novel driving violation cause analysis (DVCA) tool. Our tool can achieve perfect component-level attribution accuracy (100%) and almost (>98%) perfect message-level accuracy.
  arXiv  Detail & Related papers  (2024-01-19T01:12:37Z)
• Unsupervised Domain Adaptation for Self-Driving from Past Traversal Features [69.47588461101925]
  We propose a method to adapt 3D object detectors to new driving environments. Our approach enhances LiDAR-based detection models using spatial quantized historical features. Experiments on real-world datasets demonstrate significant improvements.
  arXiv  Detail & Related papers  (2023-09-21T15:00:31Z)
• AutoFed: Heterogeneity-Aware Federated Multimodal Learning for Robust Autonomous Driving [15.486799633600423]
  AutoFed is a framework to fully exploit multimodal sensory data on autonomous vehicles. We propose a novel model leveraging pseudo-labeling to avoid mistakenly treating unlabeled objects as the background. We also propose an autoencoder-based data imputation method to fill missing data modality.
  arXiv  Detail & Related papers  (2023-02-17T01:31:53Z)
• Learning energy-efficient driving behaviors by imitating experts [75.12960180185105]
  This paper examines the role of imitation learning in bridging the gap between control strategies and realistic limitations in communication and sensing. We show that imitation learning can succeed in deriving policies that, if adopted by 5% of vehicles, may boost the energy-efficiency of networks with varying traffic conditions by 15% using only local observations.
  arXiv  Detail & Related papers  (2022-06-28T17:08:31Z)
• Transferable Deep Reinforcement Learning Framework for Autonomous Vehicles with Joint Radar-Data Communications [69.24726496448713]
  We propose an intelligent optimization framework based on the Markov Decision Process (MDP) to help the AV make optimal decisions. We then develop an effective learning algorithm leveraging recent advances of deep reinforcement learning techniques to find the optimal policy for the AV. We show that the proposed transferable deep reinforcement learning framework reduces the obstacle miss detection probability by the AV up to 67% compared to other conventional deep reinforcement learning approaches.
  arXiv  Detail & Related papers  (2021-05-28T08:45:37Z)
• Improving Robustness of Learning-based Autonomous Steering Using Adversarial Images [58.287120077778205]
  We introduce a framework for analyzing robustness of the learning algorithm w.r.t varying quality in the image input for autonomous driving. Using the results of sensitivity analysis, we propose an algorithm to improve the overall performance of the task of "learning to steer"
  arXiv  Detail & Related papers  (2021-02-26T02:08:07Z)
• Detecting 32 Pedestrian Attributes for Autonomous Vehicles [103.87351701138554]
  In this paper, we address the problem of jointly detecting pedestrians and recognizing 32 pedestrian attributes. We introduce a Multi-Task Learning (MTL) model relying on a composite field framework, which achieves both goals in an efficient way. We show competitive detection and attribute recognition results, as well as a more stable MTL training.
  arXiv  Detail & Related papers  (2020-12-04T15:10:12Z)
• Towards robust sensing for Autonomous Vehicles: An adversarial perspective [82.83630604517249]
  It is of primary importance that the resulting decisions are robust to perturbations. Adversarial perturbations are purposefully crafted alterations of the environment or of the sensory measurements. A careful evaluation of the vulnerabilities of their sensing system(s) is necessary in order to build and deploy safer systems.
  arXiv  Detail & Related papers  (2020-07-14T05:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.