Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding

• URL: http://arxiv.org/abs/2608.08512v1
• Date: Sun, 09 Aug 2026 06:17:41 GMT
• Title: Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding
• Abstract summary: Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time.<n>We present TIDE, an expert-verified benchmark of 3,050 QA pairs over 644 official customs instruments issued by the Government of Bangladesh.
• Score: 3.285110528753401
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates. In contrast to encyclopedic knowledge, where an old fact is simply overwritten, an amendment is itself an official text that states what it replaces and when it takes effect, and the earlier version stays correct for its validity period. The central challenge is therefore version resolution, that is, identifying the version in force on the queried date. Existing temporal QA datasets treat time only as an annotation, so version resolution stays untested. We present TIDE, an expert-verified benchmark of 3,050 QA pairs over 644 official customs instruments issued between 1969 and 2025 by the Government of Bangladesh, covering eight task types over deeply code-mixed documents that are heterogeneous in layout and dated in two calendars. In addition, we evaluate nine recent LLMs under a single protocol across parametric, gold-context, and retrieval access, scored by a three-judge LLM council with a hard date gate separating correct meaning from correct time. The best macro-averaged accuracy is only 68.5%. Resolving a version from an implicit date reaches 59.7%, and detecting that the supplied version does not govern the query reaches only 26.7%. Models are more likely to find correct versions than to reject incorrect ones, and they tend to follow a confident parametric answer over the supplied authoritative text. All code and data are available at https://github.com/icsetepa44/TIDE

Related papers

• TimelyRAG: Semantic-Temporal Hybrid Retrieval for Time-Critical Question Answering in Overlapping-Evolving Documents [32.19368966069081]
  TimelyRAG is a retriever-agnostic framework that incorporates temporal distance into ranking to align queries with version-appropriate documents.<n>We also introduce TimelyQABench, the first benchmark for regulation-heavy domains with overlapping-evolving challenges.
  arXiv  Detail & Related papers  (2026-09-10T14:07:20Z)
• Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems [67.04448659688579]
  We develop a query-independent 16-entry codebook that ranks the top decoded candidates in a single batched forward pass.<n>With no iterative search and no message passing at test time, Codebook Agent is the most accurate method on all six benchmarks.
  arXiv  Detail & Related papers  (2026-09-02T08:10:22Z)
• From Traceability to Justifiability: Accountability Structures in Agentic Software Engineering [0.0]
  We measure, from public material only, whether AI records can express a claim and whether they hold where declared.<n>Across 188 double-graded cells we found no platform whose default record emits a content identity of a behavioral system.<n>An instrument computes realized assurance depth from a pipeline's published exhaust alone and compares it with the declared depth.
  arXiv  Detail & Related papers  (2026-08-21T16:45:00Z)
• Temporal Misgrounding in Legal RAG: A Versioned-Corpus Benchmark for French Tax Law [0.0]
  We identify and quantify temporal misgrounding: the systematic retrieval and citation of the currently in-force version of a legal article.<n>We introduce FiscalQA Pro, pairing a versioned corpus of 32,436 article-versions of the French tax code with an all-model-hard temporal-reasoning track.<n>Our end-to-end retriever over a multi-version index, with no oracle, reaches 98.3% mean strict.
  arXiv  Detail & Related papers  (2026-08-10T10:20:13Z)
• When Memory Updates but Behavior Does Not: Repairing Implicit Stale Dependencies in Personalized Agent Responses [1.5770673503612611]
  Memory-augmented agents can know that a user's stored state is outdated and still plan around the old value.<n>We identify one structural contributor: draft-anchored verification checks what a response says, and in an open-ended response the stale dependency is usually unsaid.
  arXiv  Detail & Related papers  (2026-08-03T02:49:08Z)
• Accuracy Hides How Language Models Fail: Measuring Failure States Under Matched Output Budgets [0.0]
  Language-model benchmarks collapse two distinct measurement questions into a single accuracy score.<n>We introduce a two-layer evaluation framework that separates scorer-independent execution evidence.
  arXiv  Detail & Related papers  (2026-07-27T11:04:17Z)
• HLE-Verified: A Systematic Verification and Structured Revision of Humanity's Last Exam [63.84155758655084]
  Humanity's Last Exam (HLE) has become a widely used benchmark for evaluating frontier large language models.<n>We introduce HLE-Verified, a verified and revised version of HLE with a transparent verification protocol and fine-grained error taxonomy.<n>We evaluate seven state-of-the-art language models on HLE and HLE-Verified, observing an average absolute accuracy gain of 7--10 percentage points.
  arXiv  Detail & Related papers  (2026-02-15T02:50:15Z)
• TimeStampEval: A Simple LLM Eval and a Little Fuzzy Matching Trick to Improve Search Accuracy [0.0]
  TimeStampEval is a benchmark for retrieving precise millisecond timestamps from long transcripts given non-verbatim quotes.<n>Our simple two-stage method dramatically improves retrieval accuracy while cutting inference costs by over 90%.<n>The motivating use case is an automated long-form podcast that assembles Congressional Record clips into AI-hosted verbatim narration.
  arXiv  Detail & Related papers  (2025-10-27T21:54:56Z)
• VersionRAG: Version-Aware Retrieval-Augmented Generation for Evolving Documents [2.657536539253924]
  Existing approaches achieve only 58-64% accuracy on version-sensitive questions.<n>We present VersionRAG, a version-aware RAG framework that explicitly models document evolution.<n>During retrieval, VersionRAG routes queries through specialized paths based on intent classification.
  arXiv  Detail & Related papers  (2025-10-09T11:48:58Z)
• LEXam: Benchmarking Legal Reasoning on 340 Law Exams [76.3521146499006]
  We introduce textscLEXam, a novel benchmark derived from 340 law exams spanning 116 law school courses across a range of subjects and degree levels.<n>The dataset comprises 4,886 law exam questions in English and German, including 2,841 long-form, open-ended questions and 2,045 multiple-choice questions.<n>Our results underscore the effectiveness of the dataset in differentiating between models with varying capabilities.
  arXiv  Detail & Related papers  (2025-05-19T08:48:12Z)
• Language Modeling with Editable External Knowledge [90.7714362827356]
  This paper introduces ERASE, which improves model behavior when new documents are acquired. It incrementally deletes or rewriting other entries in the knowledge base each time a document is added. It improves accuracy relative to conventional retrieval-augmented generation by 7-13% (Mixtral-8x7B) and 6-10% (Llama-3-8B) absolute.
  arXiv  Detail & Related papers  (2024-06-17T17:59:35Z)
• GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence [64.95492752484171]
  We present GenAudit -- a tool intended to assist fact-checking LLM responses for document-grounded tasks.<n>GenAudit suggests edits to the LLM response by revising or removing claims that are not supported by the reference document, and also presents evidence from the reference for facts that do appear to have support.<n> Comprehensive evaluation by human raters shows that GenAudit can detect errors in 8 different LLM outputs when summarizing documents from diverse domains.
  arXiv  Detail & Related papers  (2024-02-19T21:45:55Z)
• Factual Error Correction for Abstractive Summaries Using Entity Retrieval [57.01193722520597]
  We propose an efficient factual error correction system RFEC based on entities retrieval post-editing process. RFEC retrieves the evidence sentences from the original document by comparing the sentences with the target summary. Next, RFEC detects the entity-level errors in the summaries by considering the evidence sentences and substitutes the wrong entities with the accurate entities from the evidence sentences.
  arXiv  Detail & Related papers  (2022-04-18T11:35:02Z)
• A Dataset for Answering Time-Sensitive Questions [88.95075983560331]
  Time is an important dimension in our physical world. Lots of facts can evolve with respect to time. It is important to consider the time dimension and empower the existing QA models to reason over time. The existing QA datasets contain rather few time-sensitive questions, hence not suitable for diagnosing or benchmarking the model's temporal reasoning capability.
  arXiv  Detail & Related papers  (2021-08-13T16:42:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.