EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving

• URL: http://arxiv.org/abs/2604.22851v1
• Date: Wed, 22 Apr 2026 07:49:02 GMT
• Title: EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving
• Abstract summary: Vision-Language Models (VLMs) have advanced highlevel reasoning in autonomous driving.<n>But their ability to ground this reasoning in the underlying physics of ego-motion remains poorly understood.<n>We introduce EgoDyn-Bench, a diagnostic benchmark for evaluating the semantic ego-motion understanding of vision-centric foundation models.
• Score: 40.67852883277799
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Vision-Language Models (VLMs) have advanced highlevel reasoning in autonomous driving, their ability to ground this reasoning in the underlying physics of ego-motion remains poorly understood. We introduce EgoDyn-Bench, a diagnostic benchmark for evaluating the semantic ego-motion understanding of vision-centric foundation models. By mapping continuous vehicle kinematics to discrete motion concepts via a deterministic oracle, we decouple a model's internal physical logic from its visual perception. Our large-scale empirical audit spanning 20 + models, including closed-source MLLMs, open-source VLMs across multiple scales, and specialized VLAs, identifies a significant Perception Bottleneck: while models exhibit logical physical concepts, they consistently fail to accurately align them with visual observations, frequently underperforming classical non-learned geometric baselines. This failure persists across model scales and domain-specific training, indicating a structural deficit in how current architectures couple visual perception with physical reasoning. We demonstrate that providing explicit trajectory encodings substantially restores physical consistency across all evaluated models, revealing a functional disentanglement between vision and language: egomotion logic is derived almost exclusively from the language modality, while visual observations contribute negligible additional signal. This structural finding provides a standardized diagnostic framework and a practical pathway toward physically aligned embodied AI. Keywords: Ego-motion - Physical Reasoning - Foundation Models

Related papers

• Towards Generalizable Visually Grounded Exploration of Household Devices [74.66117923766632]
  Key bottleneck of realizing general autonomous embodied agents lies in Generalizable Visually Grounded Exploration.<n>We introduce VGEBench, a benchmark designed to evaluate the generalizable visually grounded exploration capabilities of Vision-Language Models.<n>Unlike static datasets, we construct a Logic-Driven State Machine framework. This framework simulates multi-turn interaction loops, compelling agents to achieve goals by active visual perception and feedback-driven correction.
  arXiv  Detail & Related papers  (2026-09-01T07:42:45Z)
• Explicit Kinematic Guidance from Analytic Concepts for Vision-Language-Action Models [67.98329043920803]
  Current Vision-Language-Action (VLA) models rely mainly on 2D inputs, neglecting the rich object structural information and commonsense knowledge inherent in the 3D physical world.<n>This deficiency restricts their spatial awareness and adaptability for complex, high-precision manipulation.<n>We construct a Concept Expert module for VLA to build executable Analytic Concepts that represent objects as explicit, programmatic blueprints.
  arXiv  Detail & Related papers  (2026-07-29T06:24:25Z)
• Probing Visual Concepts in Lightweight Vision-Language Models for Automated Driving [3.333320380836246]
  Vision-Language Models (VLMs) are used in automated driving applications.<n>These models often fail on simple visual questions that are highly relevant to automated driving.<n>We show that concepts such as the presence of an object or agent in a scene are explicitly and linearly encoded.<n>Other spatial visual concepts, such as the orientation of an object or agent, are only implicitly encoded by the spatial structure retained by the vision encoder.<n>Finally, we show that increasing the distance of the object in question quickly degrades the linear separability of the corresponding visual concept.
  arXiv  Detail & Related papers  (2026-03-06T09:07:57Z)
• Robust Egocentric Referring Video Object Segmentation via Dual-Modal Causal Intervention [58.05340906967343]
  Egocentric Referring Video Object (Ego-RVOS) aims to segment the specific object actively involved in a human action, as described by a language query, within first-person videos.<n>Existing methods often struggle, learning spurious correlations from skewed object-action pairings in datasets.<n>We introduce Causal-REferring (CERES), a plug-in causal framework that adapts strong, pre-trained RVOSs to the egocentric domain.
  arXiv  Detail & Related papers  (2025-12-30T16:22:14Z)
• PAI-Bench: A Comprehensive Benchmark For Physical AI [70.22914615084215]
  Video generative models often struggle to maintain physically coherent dynamics.<n>Multi-modal large language models exhibit limited performance in forecasting and causal interpretation.<n>These observations suggest that current systems are still at an early stage in handling the perceptual and predictive demands of Physical AI.
  arXiv  Detail & Related papers  (2025-12-01T18:47:39Z)
• Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmark [48.02995109011304]
  Video generation models have emerged as potential world simulators through Chain-of-Frames (CoF) reasoning.<n>Existing benchmarks, focusing on fidelity or alignment, do not assess CoF reasoning.<n>We introduce Gen-ViRe, a framework grounded in cognitive science and real-world AI applications.
  arXiv  Detail & Related papers  (2025-11-17T19:11:39Z)
• Does Physics Knowledge Emerge in Frontier Models? [19.035965618393096]
  Leading Vision-Language Models (VLMs) show strong results in visual perception and general reasoning.<n>But their ability to understand and predict physical dynamics remains unclear.<n>We benchmark six frontier VLMs on three physical simulation datasets.
  arXiv  Detail & Related papers  (2025-10-03T22:30:06Z)
• Learning Interpretable Logic Rules from Deep Vision Models [6.854329442341952]
  VisionLogic is a framework to extract interpretable logic rules from deep vision models.<n>It provides local explanations for single images and global explanations for specific classes.<n> VisionLogic also facilitates the study of visual concepts encoded by predicates.
  arXiv  Detail & Related papers  (2025-03-13T17:04:04Z)
• A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMs [3.2228025627337864]
  This paper introduces a structured evaluation framework to dissect the perception-reasoning interface in Vision-Language Models (VLMs)<n>We propose three distinct evaluation paradigms, mirroring human problem-solving strategies.<n>Applying this framework, we demonstrate that CA, leveraging powerful language models for reasoning over rich, independently generated descriptions, achieves new state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2025-01-23T12:42:42Z)
• Retrieval-Based Interleaved Visual Chain-of-Thought in Real-World Driving Scenarios [69.00444996464662]
  We propose RIV-CoT, a Retrieval-Based Interleaved Visual Chain-of-Thought method that enables vision-language models to reason using visual crops corresponding to relevant entities.<n>Our experiments demonstrate that RIV-CoT improves answer accuracy by 3.1% and reasoning accuracy by 4.6% over vanilla CoT prompting.
  arXiv  Detail & Related papers  (2025-01-08T18:31:16Z)
• ContPhy: Continuum Physical Concept Learning and Reasoning from Videos [86.63174804149216]
  ContPhy is a novel benchmark for assessing machine physical commonsense. We evaluated a range of AI models and found that they still struggle to achieve satisfactory performance on ContPhy. We also introduce an oracle model (ContPRO) that marries the particle-based physical dynamic models with the recent large language models.
  arXiv  Detail & Related papers  (2024-02-09T01:09:21Z)
• Reason induced visual attention for explainable autonomous driving [2.090380922731455]
  Deep learning (DL) based computer vision (CV) models are generally considered as black boxes due to poor interpretability. This study is motivated by the need to enhance the interpretability of DL model in autonomous driving. The proposed framework imitates the learning process of human drivers by jointly modeling the visual input (images) and natural language.
  arXiv  Detail & Related papers  (2021-10-11T18:50:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.