EASE-TTT: Evidence-Aligned Selective Test-Time Training for Long-Context Question Answering

• URL: http://arxiv.org/abs/2606.06906v1
• Date: Fri, 05 Jun 2026 04:49:37 GMT
• Title: EASE-TTT: Evidence-Aligned Selective Test-Time Training for Long-Context Question Answering
• Abstract summary: We propose Evidence-Aligned SElective Test-Time Training (EASE-TTT) for long-context question answering.<n>EASE-TTT converts selected evidence chunks into a soft attention supervision target over their token positions.<n>It achieves the strongest macro-average performance among full-context inference, retrieval-only baselines, and qTTT.
• Score: 61.89411578705886
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-context question answering (QA) remains challenging for smaller language models even when answer-bearing evidence is already present in the input. Existing within-context retrieval methods localize and expose candidate evidence chunks for the question, but they stop at input-level evidence exposure rather than adapting the query-side attention parameters that control how the model allocates attention over full-context positions. In contrast, lightweight test-time adaptation methods, such as query-only test-time training (qTTT), leave evidence localization unresolved because their generic span-level self-supervised objectives do not identify which context positions support the current answer. In this paper, we propose Evidence-Aligned SElective Test-Time Training (EASE-TTT), a within-context retrieval-augmented test-time training framework that converts selected evidence chunks into a soft attention supervision target over their token positions. Instead of replacing the full context with retrieved chunks, EASE-TTT uses the resulting attention target to guide query-side adaptation, with the adapted model generating the final answer from the original full context. Experiments on six LongBench QA tasks and three small decoder-only language models show that EASE-TTT achieves the strongest macro-average performance among full-context inference, retrieval-only baselines, and qTTT, supporting evidence-aligned test-time adaptation in long-context QA.

Related papers

• Self-Guided Test-Time Training for Long-Context LLMs [16.160114043456215]
  Long-context processing is increasingly important for large language models.<n>As input length grows, accuracy often degrades, indicating that models still struggle to identify and use the evidence most relevant to a question.<n>A promising way to improve long-context utilization is test-time training (TTT), which treats the test context as a training example for instance-specific parameter adaptation.<n>We propose Self-Guided TTT (S-TTT), which identifies the evidence spans it should learn from, and the standard language-modeling training objective is applied only to those selected spans.
  arXiv  Detail & Related papers  (2026-07-10T13:45:56Z)
• TTA-Vid: Generalized Test-Time Adaptation for Video Reasoning [54.70019148172847]
  We leverage the paradigm of Test-Time Reinforcement Learning on video-language data to allow for adapting a pretrained model to incoming video samples at test-time without explicit labels.<n>The proposed test-time adaptation for video approach (TTA-Vid) combines two components that work simultaneously.<n>TTA-Vid yields consistent improvements across various video reasoning tasks and is able to outperform current state-of-the-art methods trained on large-scale data.
  arXiv  Detail & Related papers  (2026-04-01T09:52:57Z)
• Inferential Question Answering [67.54465021408724]
  We introduce Inferential QA -- a new task that challenges models to infer answers from answer-supporting passages which provide only clues.<n>To study this problem, we construct QUIT (QUestions requiring Inference from Texts) dataset, comprising 7,401 questions and 2.4M passages.<n>We show that methods effective on traditional QA tasks struggle in inferential QA: retrievers underperform, rerankers offer limited gains, and fine-tuning provides inconsistent improvements.
  arXiv  Detail & Related papers  (2026-02-01T14:02:43Z)
• Context-Picker: Dynamic context selection using multi-stage reinforcement learning [4.539896456749749]
  We introduce emphContext-Picker, a reasoning-aware framework for long-context question answering.<n>Context-Picker treats context selection as a decision-making process optimized via a human-inspired, two-stage reinforcement learning schedule.<n>Experiments on five long-context and multi-hop QA benchmarks demonstrate that Context-Picker significantly outperforms strong RAG baselines.
  arXiv  Detail & Related papers  (2025-12-16T14:52:11Z)
• The benefits of query-based KGQA systems for complex and temporal questions in LLM era [55.20230501807337]
  Large language models excel in question-answering (QA) yet still struggle with multi-hop reasoning and temporal questions.<n> Query-based knowledge graph QA (KGQA) offers a modular alternative by generating executable queries instead of direct answers.<n>We explore multi-stage query-based framework for WikiData QA, proposing multi-stage approach that enhances performance on challenging multi-hop and temporal benchmarks.
  arXiv  Detail & Related papers  (2025-07-16T06:41:03Z)
• In-Context Learning for Pure Exploration [28.404325855738502]
  We study the problem active sequential hypothesis testing, also known as pure exploration.<n>We introduce In-Context Pure Exploration (ICPE), which meta-trains Transformers to map observation histories to query actions and a predicted hypothesis.<n>ICPE actively gathers evidence on new tasks and infers the true hypothesis without parameter updates.
  arXiv  Detail & Related papers  (2025-06-02T17:04:50Z)
• Options-Aware Dense Retrieval for Multiple-Choice query Answering [5.098112872671412]
  Long-context multiple-choice question answering tasks require robust reasoning over extensive text sources.<n>Prior research in this domain has predominantly utilized pre-trained dense retrieval models.<n>This paper proposes a novel method called Options Aware Dense Retrieval (OADR) to address these challenges.
  arXiv  Detail & Related papers  (2025-01-27T15:03:26Z)
• The Surprising Effectiveness of Test-Time Training for Few-Shot Learning [59.309477460893916]
  Language models (LMs) have shown impressive performance on tasks within their training distribution, but often struggle with structurally novel tasks.<n>We investigate the effectiveness of test-time training (TTT) as a mechanism for improving LMs' reasoning and few-shot learning capabilities.<n>Our findings highlight the limitations of in-context learning for novel tasks and demonstrate the potential of test-time training to enhance language model adaptability.
  arXiv  Detail & Related papers  (2024-11-11T18:59:45Z)
• Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack [33.178008350124315]
  We introduce Lifelong ICL, a problem setting that challenges long-context language models (LMs) to learn a sequence of language tasks through in-context learning (ICL)<n>We introduce Task Haystack, an evaluation suite dedicated to assessing and diagnosing how long-context LMs utilize contexts in Lifelong ICL.
  arXiv  Detail & Related papers  (2024-07-23T17:57:41Z)
• Detecting Multimodal Situations with Insufficient Context and Abstaining from Baseless Predictions [75.45274978665684]
  Vision-Language Understanding (VLU) benchmarks contain samples where answers rely on assumptions unsupported by the provided context.<n>We collect contextual data for each sample whenever available and train a context selection module to facilitate evidence-based model predictions.<n>We develop a general-purpose Context-AwaRe Abstention detector to identify samples lacking sufficient context and enhance model accuracy.
  arXiv  Detail & Related papers  (2024-05-18T02:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.