CounterRefine: Answer-Conditioned Counterevidence Retrieval for Inference-Time Knowledge Repair in Factual Question Answering

• URL: http://arxiv.org/abs/2603.16091v1
• Date: Tue, 17 Mar 2026 03:27:25 GMT
• Title: CounterRefine: Answer-Conditioned Counterevidence Retrieval for Inference-Time Knowledge Repair in Factual Question Answering
• Abstract summary: We present CounterRefine, a lightweight inference-time repair layer for retrieval-grounded question answering.<n>CounterRefine first produces a short answer from retrieved evidence, then gathers additional support and conflicting evidence with follow-up queries conditioned on that draft answer.<n>On the full SimpleQA benchmark, CounterRefine improves a matched GPT-5 Baseline-RAG by 5.8 points and reaches a 73.1 percent correct rate, while exceeding the reported one-shot GPT-5.4 score by roughly 40 points.
• Score: 1.1279808969568255
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In factual question answering, many errors are not failures of access but failures of commitment: the system retrieves relevant evidence, yet still settles on the wrong answer. We present CounterRefine, a lightweight inference-time repair layer for retrieval-grounded question answering. CounterRefine first produces a short answer from retrieved evidence, then gathers additional support and conflicting evidence with follow-up queries conditioned on that draft answer, and finally applies a restricted refinement step that outputs either KEEP or REVISE, with proposed revisions accepted only if they pass deterministic validation. In effect, CounterRefine turns retrieval into a mechanism for testing a provisional answer rather than merely collecting more context. On the full SimpleQA benchmark, CounterRefine improves a matched GPT-5 Baseline-RAG by 5.8 points and reaches a 73.1 percent correct rate, while exceeding the reported one-shot GPT-5.4 score by roughly 40 points. These findings suggest a simple but important direction for knowledgeable foundation models: beyond accessing evidence, they should also be able to use that evidence to reconsider and, when necessary, repair their own answers.

Related papers

• From Topical Relevance to Answerability: Entailment Distillation for Conversational Retrieval [24.372303326753535]
  We propose CLEAR, a framework that shifts conversational retrieval from topical relevance to answerability.<n>The core of CLEAR is entailment distillation, which transfers answer-passage entailment supervision into a cross-encoder reranker.<n> CLEAR consistently improves top-ranked precision over strong query-rewriting and dense-retrieval baselines.
  arXiv  Detail & Related papers  (2026-09-03T07:38:36Z)
• SEER: A Self-Grounded Evidence Interface for Controlled Spatial Relation Classification [71.9783246097687]
  SEER hides candidate relations during pair localization, constructs a query-specific view with explicit subject/object roles, and retains the full image and sparse box geometry as complementary evidence.<n>For relation-choice protocols with exact inverse support, an optional refinement swaps the entity roles and changes the forward decision only when exactly one visual state obeys the corresponding inverse relation.<n>The unchanged protocol yields +4.35 to +11.79 on all 2,434 filtered EmbSpatial pair-relation questions across three models.
  arXiv  Detail & Related papers  (2026-08-04T13:16:15Z)
• Retroactive Chain-of-Thought (RetroCoT): Forensic Reconstruction Prompts as a Safety Diagnostic Across Model Generations [0.0]
  We show that current alignment policies are not invariant to semantic equivalence.<n>We introduce Retroactive Chain-of-Thought (RetroCoT), a single-turn attack that reframes harmful requests as forensic reconstruction tasks.
  arXiv  Detail & Related papers  (2026-07-06T03:59:06Z)
• QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards [68.15832368034815]
  We present a framework that co-designs queries and rubrics.<n>QUBRIC achieves a +5.5 point gain on ArenaHard over the SFT baseline.
  arXiv  Detail & Related papers  (2026-06-02T17:53:04Z)
• PAVE: Premise-Aware Validation and Editing for Retrieval-Augmented LLMs [8.082352336629816]
  PAVE is an inference-time validation layer for evidence-grounded question answering.<n>It decomposes retrieved context into question-conditioned atomic facts, drafts an answer, scores how well that draft is supported by the extracted premises, and revises low-support outputs before finalization.
  arXiv  Detail & Related papers  (2026-03-21T06:23:21Z)
• Inferential Question Answering [67.54465021408724]
  We introduce Inferential QA -- a new task that challenges models to infer answers from answer-supporting passages which provide only clues.<n>To study this problem, we construct QUIT (QUestions requiring Inference from Texts) dataset, comprising 7,401 questions and 2.4M passages.<n>We show that methods effective on traditional QA tasks struggle in inferential QA: retrievers underperform, rerankers offer limited gains, and fine-tuning provides inconsistent improvements.
  arXiv  Detail & Related papers  (2026-02-01T14:02:43Z)
• QUARK: Robust Retrieval under Non-Faithful Queries via Query-Anchored Aggregation [2.505352949111876]
  QUARK is a training-free framework for robust retrieval under non-faithful queries.<n>The design enables QUARK to improve recall and ranking quality without sacrificing robustness.
  arXiv  Detail & Related papers  (2026-01-28T21:14:49Z)
• Retrieve-Refine-Calibrate: A Framework for Complex Claim Fact-Checking [32.6738019397553]
  We propose a Retrieve-Refine-Calibrate (RRC) framework based on large language models (LLMs)<n>Specifically, the framework first identifies the entities mentioned in the claim and retrieves evidence relevant to them.<n>Then, it refines the retrieved evidence based on the claim to reduce irrelevant information.<n>Finally, it calibrates the verification process by re-evaluating low-confidence predictions.
  arXiv  Detail & Related papers  (2026-01-23T08:48:52Z)
• Adaptive Retrieval for Reasoning-Intensive Retrieval [60.30588731127791]
  Bridge documents are those that contribute to the reasoning process but are not directly relevant to the initial query.<n>Existing reasoning-based reranker pipelines attempt to surface these documents in ranking, they suffer from bounded recall.<n>We propose a framework that bridges this gap by repurposing reasoning plans as dense feedback signals for adaptive retrieval.
  arXiv  Detail & Related papers  (2026-01-08T05:46:50Z)
• NeoQA: Evidence-based Question Answering with Generated News Events [53.85274258429368]
  We introduce NeoQA, a benchmark designed to address this issue.<n>We propose our dataset as a new platform for evaluating evidence-based question answering.
  arXiv  Detail & Related papers  (2025-05-09T10:51:29Z)
• Retrieval-Augmented Generation by Evidence Retroactivity in LLMs [19.122314663040726]
  Retroactive Retrieval-Augmented Generation (RetroRAG) is a novel framework to build a retroactive reasoning paradigm.<n>RetroRAG revises and updates the evidence, redirecting the reasoning chain to the correct direction.<n> Empirical evaluations show that RetroRAG significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2025-01-07T08:57:42Z)
• Atomic Fact Decomposition Helps Attributed Question Answering [29.67882325906939]
  Attributed Question Answering (AQA) aims to provide both a trustworthy answer and a reliable attribution report for a question.<n>This paper proposes an Atomic fact decomposition-based Retrieval and Editing framework.<n>It decomposes the generated long-form answers into molecular clauses and atomic facts by the instruction-tuned LLMs.
  arXiv  Detail & Related papers  (2024-10-22T05:25:54Z)
• Contrastive Learning to Improve Retrieval for Real-world Fact Checking [84.57583869042791]
  We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for fact-checking complex claims. We leverage the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents. We find a 6% improvement in veracity classification accuracy on the dataset.
  arXiv  Detail & Related papers  (2024-10-07T00:09:50Z)
• Localizing and Mitigating Errors in Long-form Question Answering [79.63372684264921]
  Long-form question answering (LFQA) aims to provide thorough and in-depth answers to complex questions, enhancing comprehension.<n>This work introduces HaluQuestQA, the first hallucination dataset with localized error annotations for human-written and model-generated LFQA answers.
  arXiv  Detail & Related papers  (2024-07-16T17:23:16Z)
• From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verification [118.03466985807331]
  We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence.<n>We introduce the feedback-based evidence retriever(FER) that optimize the evidence retrieval process by incorporating feedback from the claim verifier.
  arXiv  Detail & Related papers  (2023-10-18T02:59:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.