Evaluating Temporal Consistency in Multi-Turn Language Models

• URL: http://arxiv.org/abs/2604.23051v1
• Date: Fri, 24 Apr 2026 22:44:21 GMT
• Title: Evaluating Temporal Consistency in Multi-Turn Language Models
• Abstract summary: Language models are increasingly deployed in interactive settings where users reason about facts over time rather than in isolation.<n>We study this challenge through the lens of temporal scope stability.<n>We introduce ChronoScope, a large-scale diagnostic benchmark designed to isolate temporal scope behavior in controlled multi-turn interactions.
• Score: 5.535674707194489
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language models are increasingly deployed in interactive settings where users reason about facts over time rather than in isolation. In such scenarios, correct behavior requires models to maintain and update implicit temporal assumptions established earlier in a conversation. We study this challenge through the lens of temporal scope stability: the ability to preserve, override, or transfer time-scoped factual context across dialogue turns. We introduce ChronoScope, a large-scale diagnostic benchmark designed to isolate temporal scope behavior in controlled multi-turn interactions, comprising over one million deterministically generated question chains grounded in Wikidata. ChronoScope evaluates whether models can correctly retain inferred temporal scope when follow-up questions omit explicit time references, spanning implicit carryover, explicit scope switching, cross-entity transfer, and longer temporal trajectories. Through extensive evaluation of state-of-the-art language models, we find that temporal scope stability is frequently violated in controlled multi-turn settings, with models often drifting toward present-day assumptions despite correct underlying knowledge. These failures intensify with interaction length and persist even under oracle context conditions, revealing a gap between single-turn factual accuracy and coherent temporal reasoning under sequential interaction. We make our dataset and evaluation suite publicly available at https://github.com/yashkumaratri/ChronoScope

Related papers

• What's the Catch? Evaluating Temporal Consistency in Vision-Language Models [0.11121828451787184]
  Vision-language models (VLMs) achieve strong performance on video and image-sequence benchmarks, yet it remains unclear whether they capture temporal structure.<n>We introduce TimeCatch, where temporal anomalies are created by swapping consecutive frames and frame-level anomalies by replacing a frame with Gaussian noise.<n>Our evaluation reveals a substantial gap between frame-level and temporal anomaly detection.
  arXiv  Detail & Related papers  (2026-08-24T16:40:49Z)
• The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context [76.36635039699112]
  Large language models (LLMs) are increasingly deployed in context-rich settings where task inputs are often accompanied by long, partially irrelevant context.<n>We find that state-of-the-art models often appear robust to task-irrelevant context at the aggregate level.<n>This aggregate stability, however, masks significant per-example instability.
  arXiv  Detail & Related papers  (2026-07-14T17:01:12Z)
• Beyond Static Dialogues: Benchmarking Realistic, Heterogeneous, and Evolving Long-Term Memory [54.947805187562274]
  We introduce realistic dialogues across diverse interaction scenarios that exhibit dynamic temporal evolution and long-term coherence.<n>The resulting benchmark encompasses challenging question-answer pairs spanning seven inquiry types.<n>We identify at least one of 27 critical memory characteristics that we identify as essential yet underexplored in current research.
  arXiv  Detail & Related papers  (2026-05-29T09:51:22Z)
• Tracking the Truth: Object-Centric Spatio-Temporal Monitoring for Video Large Language Models [154.39583176906893]
  Large language models (MLs) have advanced understanding, they remain highly prone to hallucinations in scenes.<n>We argue this stems from a failure intemporal monitoring, the ability to persistently track object dynamic identities, states, and relations over time.<n>Existing benchmarks obscure this deficit relying on single final-answer queries that can often be resolved via local visual cues or statistical priors.
  arXiv  Detail & Related papers  (2026-05-09T14:32:36Z)
• From Observations to States: Latent Time Series Forecasting [65.98504021691666]
  We propose Latent Time Series Forecasting (LatentTSF), a novel paradigm that shifts TSF from observation regression to latent state prediction.<n>Specifically, LatentTSF employs an AutoEncoder to project observations at each time step into a higher-dimensional latent state space.<n>Our proposed latent objectives implicitly maximize mutual information between predicted latent states and ground-truth states and observations.
  arXiv  Detail & Related papers  (2026-01-30T20:39:44Z)
• Not in Sync: Unveiling Temporal Bias in Audio Chat Models [59.146710538620816]
  Large Audio Language Models (LALMs) are increasingly applied to audio understanding and multimodal reasoning.<n>We present the first systematic study of temporal bias in LALMs, revealing a key limitation in their timestamp prediction.
  arXiv  Detail & Related papers  (2025-10-14T06:29:40Z)
• Time-RA: Towards Time Series Reasoning for Anomaly with LLM Feedback [55.284574165467525]
  Time-series Reasoning for Anomaly (Time-RA) transforms classical time series anomaly detection into a generative, reasoning-intensive task.<n>Also, we introduce the first real-world multimodal benchmark dataset, RATs40K, explicitly annotated for anomaly reasoning.
  arXiv  Detail & Related papers  (2025-07-20T18:02:50Z)
• Counterfactual-Consistency Prompting for Relative Temporal Understanding in Large Language Models [24.586475741345616]
  We tackle the issue of temporal inconsistency in large language models (LLMs) by proposing a novel counterfactual prompting approach.<n>Our method generates counterfactual questions and enforces collective constraints, enhancing the model's consistency.<n>We evaluate our method on multiple datasets, demonstrating significant improvements in event ordering for explicit and implicit events and temporal commonsense understanding.
  arXiv  Detail & Related papers  (2025-02-17T04:37:07Z)
• StreamLTS: Query-based Temporal-Spatial LiDAR Fusion for Cooperative Object Detection [0.552480439325792]
  We propose Time-Aligned COoperative Object Detection (TA-COOD), for which we adapt widely used dataset OPV2V and DairV2X. Experiment results confirm the superior efficiency of our fully sparse framework compared to the state-of-the-art dense models.
  arXiv  Detail & Related papers  (2024-07-04T10:56:10Z)
• From Link Prediction to Forecasting: Addressing Challenges in Batch-based Temporal Graph Learning [0.716879432974126]
  We show that the suitability of common batch-oriented evaluation depends on the datasets' characteristics.<n>For continuous-time temporal graphs, fixed-size batches create time windows with different durations, resulting in an inconsistent dynamic link prediction task.<n>For discrete-time temporal graphs, the sequence of batches can additionally introduce temporal dependencies that are not present in the data.
  arXiv  Detail & Related papers  (2024-06-07T12:45:12Z)
• On the Identification of Temporally Causal Representation with Instantaneous Dependence [50.14432597910128]
  Temporally causal representation learning aims to identify the latent causal process from time series observations. Most methods require the assumption that the latent causal processes do not have instantaneous relations. We propose an textbfIDentification framework for instantanetextbfOus textbfLatent dynamics.
  arXiv  Detail & Related papers  (2024-05-24T08:08:05Z)
• Instructed Diffuser with Temporal Condition Guidance for Offline Reinforcement Learning [71.24316734338501]
  We propose an effective temporally-conditional diffusion model coined Temporally-Composable diffuser (TCD) TCD extracts temporal information from interaction sequences and explicitly guides generation with temporal conditions. Our method reaches or matches the best performance compared with prior SOTA baselines.
  arXiv  Detail & Related papers  (2023-06-08T02:12:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.