Bridging Structure and Language: Graph-Based Visual Reasoning for Autonomous Road Understanding

• URL: http://arxiv.org/abs/2605.20942v1
• Date: Wed, 20 May 2026 09:28:06 GMT
• Title: Bridging Structure and Language: Graph-Based Visual Reasoning for Autonomous Road Understanding
• Abstract summary: Combined Road Substrate (CRS) is a graph-grounded framework that makes geometric road structure and open-vocabulary semantics jointly executable in a single representation.<n>CRS enables the automatic generation of compositionally complex and linguistically varied question-answer pairs.<n>We demonstrate that state-of-the-art VLMs - including large, closed-source models - struggle significantly with structured road reasoning.
• Score: 21.706539694985164
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Structured road understanding of lane geometry, topology, and traffic element relationships is foundational to safe autonomous driving. While vision-language models (VLMs) offer promising semantic flexibility, they lack the geometric and relational grounding required for precise road reasoning. Conversely, traditional modular systems, e.g., HD maps and topological road graphs, provide structural precision but remain semantically rigid. To bridge this gap, we introduce the Combined Road Substrate (CRS), a graph-grounded framework that makes geometric road structure and open-vocabulary semantics jointly executable in a single representation. CRS enables the automatic generation of compositionally complex and linguistically varied question-answer pairs via recursive graph queries, augmented with a "grounding for free" mechanism that ensures logical traceability to specific map elements, and procedurally extracted chain-of-thought supervision traces. We demonstrate that state-of-the-art VLMs - including large, closed-source models - struggle significantly with structured road reasoning, yet training a small 2- or 4-billion-parameter model with as few as 20 to 80 CRS-enriched scenes yields stable gains in compositional reasoning tasks of varying depth. Analysis of model behavior via verifiable reasoning traces reveals a systematic shift in failure modes: whereas baseline models fail at relational scene understanding, CRS-trained models reduce failures to attribute recognition, suggesting that the primary bottleneck in road understanding is not model scale, but the absence of structured supervision.

Related papers

• LeCropFollow: Latent Space Planning for Navigation in Unstructured Crop Fields [7.898784066160524]
  We present LeCropFollow, a visual navigation framework that bypasses explicit geometric modeling in favor of a learned latent representation.<n>We show that LeCropFollow matches state-of-the-art baselines in unstructured rows but significantly outperforms them in plantation gaps.<n>These results suggest that latent planning offers a robust alternative to geometric estimation for operations in heterogeneous agricultural environments.
  arXiv  Detail & Related papers  (2026-06-30T16:47:08Z)
• A Unified Framework for Structured Flow Modeling: From Continuous Fields to Data-Driven Representations [0.0]
  dynamical systems can be described in terms of structured flows combining source/sink behavior, cyclic dynamics, and topology-constrained transport.<n>This work provides a unified perspective on such systems by connecting continuous formulations based on the Helmholtz-Hodge decomposition with discrete and data-driven representations.
  arXiv  Detail & Related papers  (2026-05-18T11:52:09Z)
• Overcoming the Impedance Mismatch: A Theoretical Roadmap for Fusing Foundation Models and Knowledge Graphs [0.0]
  We formalize the underlying structural and geometric friction as the textitImpedance Mismatch.<n>We propose a rigorous theoretical roadmap to achieve true semantic fusion.<n>This framework paves the way for models that seamlessly fuse the precision of symbolic logic with the expressivity of parametric memory.
  arXiv  Detail & Related papers  (2026-04-21T17:30:15Z)
• SGR3 Model: Scene Graph Retrieval-Reasoning Model in 3D [51.32219731589742]
  3D scene graphs provide a structured representation of object entities and their relationships.<n>Existing approaches for 3D scene graph generation typically combine scene reconstruction with graph neural networks (GNNs)<n>In this work, we introduce a Scene Graph Retrieval-Reasoning Model in 3D (SGR3 Model)
  arXiv  Detail & Related papers  (2026-03-04T21:19:54Z)
• StructAlign: Structured Cross-Modal Alignment for Continual Text-to-Video Retrieval [75.28673512571449]
  A key challenge in Continual Text-to-Video Retrieval is feature drift.<n>We propose StructAlign, a structured cross-modal alignment method for CTVR.<n>Our method consistently outperforms state-of-the-art continual retrieval approaches.
  arXiv  Detail & Related papers  (2026-01-28T13:34:44Z)
• RoadSceneBench: A Lightweight Benchmark for Mid-Level Road Scene Understanding [8.824330786267184]
  RoadSceneBench is a benchmark designed to evaluate and advance visual reasoning in complex road environments.<n>Unlike large-scale perception, RoadSceneBench emphasizes relational understanding and structural consistency.<n>Our method achieves state-of-the-art performance across diverse road configurations.
  arXiv  Detail & Related papers  (2025-11-27T13:57:31Z)
• Cognitive Maps in Language Models: A Mechanistic Analysis of Spatial Planning [2.1115884707107715]
  We train GPT-2 models on three spatial learning paradigms in grid environments.<n>Using behavioural, representational and mechanistic analyses, we uncover two fundamentally different learned algorithms.
  arXiv  Detail & Related papers  (2025-11-17T13:46:19Z)
• RelTopo: Multi-Level Relational Modeling for Driving Scene Topology Reasoning [74.58385332488227]
  Road topology reasoning is critical for autonomous driving, enabling effective navigation and adherence to traffic regulations.<n>Existing methods typically focus on either lane detection or Lane-to-Lane (L2L) topology reasoning, often textitneglecting Lane-to-Traffic-element (L2T) relationships to optimize these tasks jointly.<n>We argue that relational modeling is beneficial for both perception and reasoning, as humans naturally leverage contextual relationships for road element recognition and their connectivity inference.
  arXiv  Detail & Related papers  (2025-06-16T14:40:28Z)
• On the Diagram of Thought [20.805936414171892]
  Large Language Models (LLMs) excel at many tasks but often falter on complex problems that require structured, multi-step reasoning.<n>We introduce the Diagram of Thought (DoT), a new framework that enables a single LLM to build and navigate a mental map of its reasoning.
  arXiv  Detail & Related papers  (2024-09-16T07:01:41Z)
• Learning to Model Graph Structural Information on MLPs via Graph Structure Self-Contrasting [50.181824673039436]
  We propose a Graph Structure Self-Contrasting (GSSC) framework that learns graph structural information without message passing. The proposed framework is based purely on Multi-Layer Perceptrons (MLPs), where the structural information is only implicitly incorporated as prior knowledge. It first applies structural sparsification to remove potentially uninformative or noisy edges in the neighborhood, and then performs structural self-contrasting in the sparsified neighborhood to learn robust node representations.
  arXiv  Detail & Related papers  (2024-09-09T12:56:02Z)
• Decentralized Vehicle Coordination: The Berkeley DeepDrive Drone Dataset and Consensus-Based Models [76.32775745488073]
  We present a novel dataset and modeling framework designed to study motion planning in understructured environments.<n>We demonstrate that a consensus-based modeling approach can effectively explain the emergence of priority orders observed in our dataset.
  arXiv  Detail & Related papers  (2022-09-19T05:06:57Z)
• Structural Landmarking and Interaction Modelling: on Resolution Dilemmas in Graph Classification [50.83222170524406]
  We study the intrinsic difficulty in graph classification under the unified concept of resolution dilemmas'' We propose SLIM'', an inductive neural network model for Structural Landmarking and Interaction Modelling.
  arXiv  Detail & Related papers  (2020-06-29T01:01:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.