From Prompts to Pavement Through Time: Temporal Grounding in Agentic Scene-to-Plan Reasoning

• URL: http://arxiv.org/abs/2605.19824v1
• Date: Tue, 19 May 2026 13:18:35 GMT
• Title: From Prompts to Pavement Through Time: Temporal Grounding in Agentic Scene-to-Plan Reasoning
• Abstract summary: This work explores whether temporal conditioning within inter-agent communication can preserve or enhance coherence without introducing degradation in semantic or logical consistency.<n>Results show that while temporal conditioning reshapes reasoning style, it yields no statistically significant improvements in standard NLP-based correctness metrics.<n>These findings clarify the limits of prompt-based temporal grounding and establish the first empirical benchmark for temporal scene-to-plan reasoning.
• Score: 1.2777067998526854
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent attempts to support high-level scene interpretation and planning in Autonomous Vehicles (AVs) using ensembles of Large Language Models (LLMs) and Large Multimodal Models (LMMs) continue to treat time as a secondary property. This lack of temporal grounding leads to inconsistencies in reasoning about continuous actions, undermining both safety and interpretability. This work explores whether temporal conditioning within inter-agent communication can preserve or enhance coherence without introducing degradation in semantic or logical consistency. To investigate this, we introduce three planner architectures with progressively increasing temporal integration and evaluate them on curated subsets of the BDD-X dataset using semantic, syntactic, and logical metrics. Results show that while temporal conditioning reshapes reasoning style, it yields no statistically significant improvements in standard NLP-based correctness metrics. However, qualitative analysis reveals predictive hazard reasoning, stable corrective behavior, and strategic divergence in the Sentinel. These findings clarify the limits of prompt-based temporal grounding and establish the first empirical benchmark for temporal scene-to-plan reasoning.

Related papers

• NVExplain: Explaining Time Series Forecasting with Latent Trajectory Analysis and Structure-Preserving Surrogates [2.0320459617688793]
  Time series forecasting models are widely used in high-stakes settings.<n> existing post-hoc methods often ignore temporal dependence and fail to provide horizon-specific explanations.<n>We propose a model-agnostic explainability framework that explains forecasting predictions by attributing each forecast horizon to temporally relevant historical lags.
  arXiv  Detail & Related papers  (2026-08-25T19:14:12Z)
• Learning Structural Convergence: A Neuro-Symbolic Benchmark for Temporal Reasoning [2.7644691581661287]
  This paper introduces TRACTA, a benchmark for temporal structural reasoning in high-complexity event-driven systems.<n>The benchmark includes three tasks: early_warning, pattern_detection, and run_classification.<n>Results show that raw event-level learning remains informative, but learned temporal modeling over semantic capability achieves the highest aggregate point estimates.
  arXiv  Detail & Related papers  (2026-07-24T14:50:19Z)
• Trajectory-Aware Retrieval Agents for Temporal Decision- Making [5.789945004686575]
  We study the problem of decision-making from long-form, temporally structured text using large language model (LLM) agents.<n>We introduce TLM, a closed-loop agentic framework that iteratively refines the evidence set using SHAP-guided feedback.<n> Empirically, TLM is evaluated on three temporally grounded decision tasks: medical question answering, earnings call surprise prediction, and overnight stock gap prediction.
  arXiv  Detail & Related papers  (2026-07-07T16:28:38Z)
• Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models [154.39583176906893]
  Large language models (MLs) have advanced understanding, they remain highly prone to hallucinations in scenes.<n>We argue this stems from a failure intemporal monitoring, the ability to persistently track object dynamic identities, states, and relations over time.<n>Existing benchmarks obscure this deficit relying on single final-answer queries that can often be resolved via local visual cues or statistical priors.
  arXiv  Detail & Related papers  (2026-05-09T14:32:36Z)
• Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Models [0.0]
  Internal representations carry correctness-related signals, but their coarse aggregations may obscure the token and layer structure underlying reasoning.<n>We investigate hidden-state transitions across decoding steps and layers, and identify a pattern in large reasoning models (LRMs)<n>We formalize this characteristic as Stemporal Amplitude of Latent Transition (StALT), a statistic trajectory that summarizes temporal changes between adjacent tokens weighted by within-token layer saliency.
  arXiv  Detail & Related papers  (2026-05-03T12:46:41Z)
• Probing the Reliability of Driving VLMs: From Inconsistent Responses to Grounded Temporal Reasoning [17.08518699175473]
  We investigate whether Vision-Language Models (VLMs), when applied as driving assistants, can response consistantly and understand how present observations shape future outcomes.<n>We find that models with strong visual understanding do not necessarily perform best on tasks requiring temporal reasoning.<n>We propose a simple yet effective self-supervised tuning approach with chain-of-thought reasoning that improves both consistency and temporal reasoning without requiring temporal labels.
  arXiv  Detail & Related papers  (2026-03-10T11:12:28Z)
• Hit-RAG: Learning to Reason with Long Contexts via Preference Alignment [97.55382322103986]
  Hit-RAG is a multi-stage preference alignment framework designed to resolve cognitive bottlenecks.<n>Our approach systematically refines the utilization of external evidence via three distinct stages.
  arXiv  Detail & Related papers  (2026-03-07T04:05:27Z)
• Agentic Spatio-Temporal Grounding via Collaborative Reasoning [80.83158605034465]
  Temporal Video Grounding aims to retrieve thetemporal tube of a target object or person in a video given a text query.<n>We propose the Agentic Spatio-Temporal Grounder (ASTG) framework for the task of STVG towards an open-world and training-free scenario.<n>Specifically, two specialized agents SRA (Spatial Reasoning Agent) and TRA (Temporal Reasoning Agent) constructed leveraging on modern Multimoal Large Language Models (MLLMs)<n>Experiments on popular benchmarks demonstrate the superiority of the proposed approach where it outperforms existing weakly-supervised and zero-shot approaches by a margin
  arXiv  Detail & Related papers  (2026-02-10T10:16:27Z)
• Priors in Time: Missing Inductive Biases for Language Model Interpretability [58.07412640266836]
  We show that Sparse Autoencoders impose priors that assume independence of concepts across time, implying stationarity.<n>We introduce a new interpretability objective -- Temporal Feature Analysis -- which possesses a temporal inductive bias to decompose representations at a given time into two parts.<n>Our results underscore the need for inductive biases that match the data in designing robust interpretability tools.
  arXiv  Detail & Related papers  (2025-11-03T18:43:48Z)
• Generic Temporal Reasoning with Differential Analysis and Explanation [61.96034987217583]
  We introduce a novel task named TODAY that bridges the gap with temporal differential analysis. TODAY evaluates whether systems can correctly understand the effect of incremental changes. We show that TODAY's supervision style and explanation annotations can be used in joint learning.
  arXiv  Detail & Related papers  (2022-12-20T17:40:03Z)
• Interpretable Time-series Representation Learning With Multi-Level Disentanglement [56.38489708031278]
  Disentangle Time Series (DTS) is a novel disentanglement enhancement framework for sequential data. DTS generates hierarchical semantic concepts as the interpretable and disentangled representation of time-series. DTS achieves superior performance in downstream applications, with high interpretability of semantic concepts.
  arXiv  Detail & Related papers  (2021-05-17T22:02:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.