"I Don't Know" -- Towards Appropriate Trust with Certainty-Aware Retrieval Augmented Generation

• URL: http://arxiv.org/abs/2605.00957v1
• Date: Fri, 01 May 2026 12:49:13 GMT
• Title: "I Don't Know" -- Towards Appropriate Trust with Certainty-Aware Retrieval Augmented Generation
• Abstract summary: We develop CERTA (Certainty Enhanced RAG for Trustworthy Answers), a specialized Retrieval Augmented Generation (RAG) system that incorporates the relevance between question, context, and answer to reflect its uncertainty in answering questions.<n>Our evaluations indicate that CERTA helps identify answers that are uncertain, decreases the cases of over-agreeing, and provides cautious behavior when prompted for moral judgments.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Achieving the right amount of trust in AI systems is important, but challenging. The problem is exacerbated with the rise of Large Language Models (LLMs) as they provide human-level communication capabilities, but potentially hallucinate in the content that they generate. Moreover, they express over-confidence in their answers, making it difficult for users to judge their truthfulness. An important human value that users seek is benevolence, which can be met by LLM's self-reflection leading to reliable and honest answers. Accordingly, this paper proposes conveying appropriate levels of self-reflected certainty to build appropriate trust. Our contributions are twofold: 1) We develop CERTA (Certainty Enhanced RAG for Trustworthy Answers), a specialized Retrieval Augmented Generation (RAG) system that incorporates the relevance between question, context, and answer to reflect its uncertainty in answering questions; 2) We create the Certainty Benchmark with 90 question-context pairs of non-objective questions, divided over four categories (factuality, preference, sycophancy, morality) and three types of contexts (relevant, incomplete, irrelevant). We run experiments with a baseline RAG system and three CERTA settings using two LLMs. Our evaluations indicate that CERTA helps identify answers that are uncertain, decreases the cases of over-agreeing, and provides cautious behavior when prompted for moral judgments.

Related papers

• A Tri-Agent Framework for Evaluating and Aligning Question Clarification Capabilities of Large Language Models [2.332848751256598]
  Large Language Models (LLMs) are increasingly deployed in interactive systems where understanding user intent precisely is paramount.<n>This paper introduces a novel tri-agent framework for the robust evaluation of an LLM's ability to engage in clarifying dialogue.
  arXiv  Detail & Related papers  (2026-09-02T03:33:40Z)
• Trust Me on This: A User Study of Trustworthiness for RAG Responses [15.029309551125962]
  This study investigates how different types of explanations can influence user trust in responses from retrieval-augmented generation systems.<n>Users' judgments are also heavily influenced by response clarity, actionability, and their own prior knowledge.
  arXiv  Detail & Related papers  (2026-01-20T20:30:33Z)
• SimpleDevQA: Benchmarking Large Language Models on Development Knowledge QA [58.75982433502236]
  The Dev Knowledge QA task accounts for 39.6% of interactions.<n>Only 27.5% of real Dev Knowledge QA dialogues focus on code understanding.<n>Only 17.1% of real-world Dev Knowledge QA dialogues can be used for constructing a benchmark.
  arXiv  Detail & Related papers  (2025-12-09T17:58:36Z)
• Teaching Language Models To Gather Information Proactively [53.85419549904644]
  Large language models (LLMs) are increasingly expected to function as collaborative partners.<n>In this work, we introduce a new task paradigm: proactive information gathering.<n>We design a scalable framework that generates partially specified, real-world tasks, masking key information.<n>Within this setup, our core innovation is a reinforcement finetuning strategy that rewards questions that elicit genuinely new, implicit user information.
  arXiv  Detail & Related papers  (2025-07-28T23:50:09Z)
• Harnessing RLHF for Robust Unanswerability Recognition and Trustworthy Response Generation in LLMs [2.217239320172707]
  This paper introduces Self-Aware LLM for Unanswerability (SALU), a novel approach that deeply integrates unanswerability detection directly within the generative process.<n>SALU is trained using a multi-task learning framework for both standard Question Answering (QA) and explicit abstention generation for unanswerable queries.<n>It consistently outperforms strong baselines, including hybrid LLM-classifier systems, in overall accuracy for correctly answering or abstaining from questions.
  arXiv  Detail & Related papers  (2025-07-22T18:44:18Z)
• Query-Level Uncertainty in Large Language Models [39.59641844929696]
  We propose a method to detect knowledge boundaries via Query-Level Uncertainty.<n>This method estimates if a model is capable of answering a given query before generating any tokens, thus avoiding the generation cost.<n>We demonstrate its benefits in adaptive inference settings, showing that for RAG and model cascading it reduces inference costs while preserving overall performance.
  arXiv  Detail & Related papers  (2025-06-11T12:39:48Z)
• Know3-RAG: A Knowledge-aware RAG Framework with Adaptive Retrieval, Generation, and Filtering [9.814926166669366]
  We propose Know3-RAG, a knowledge-aware RAG framework that leverages structured knowledge from knowledge graphs (KGs) to guide three core stages of the RAG process, including retrieval, generation, and filtering.<n> Experiments on multiple open-domain QA benchmarks demonstrate that Know3-RAG consistently outperforms strong baselines, significantly reducing hallucinations and enhancing answer reliability.
  arXiv  Detail & Related papers  (2025-05-19T03:25:18Z)
• REVAL: A Comprehension Evaluation on Reliability and Values of Large Vision-Language Models [59.445672459851274]
  REVAL is a comprehensive benchmark designed to evaluate the textbfREliability and textbfVALue of Large Vision-Language Models.<n>REVAL encompasses over 144K image-text Visual Question Answering (VQA) samples, structured into two primary sections: Reliability and Values.<n>We evaluate 26 models, including mainstream open-source LVLMs and prominent closed-source models like GPT-4o and Gemini-1.5-Pro.
  arXiv  Detail & Related papers  (2025-03-20T07:54:35Z)
• Do RAG Systems Cover What Matters? Evaluating and Optimizing Responses with Sub-Question Coverage [74.70255719194819]
  We introduce a novel framework based on sub-question coverage, which measures how well a RAG system addresses different facets of a question. We use this framework to evaluate three commercial generative answer engines: You.com, Perplexity AI, and Bing Chat. We find that while all answer engines cover core sub-questions more often than background or follow-up ones, they still miss around 50% of core sub-questions.
  arXiv  Detail & Related papers  (2024-10-20T22:59:34Z)
• Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance [73.19687314438133]
  We study how reliance is affected by contextual features of an interaction. We find that contextual characteristics significantly affect human reliance behavior. Our results show that calibration and language quality alone are insufficient in evaluating the risks of human-LM interactions.
  arXiv  Detail & Related papers  (2024-07-10T18:00:05Z)
• Iterative Utility Judgment Framework via LLMs Inspired by Relevance in Philosophy [66.95501113584541]
  We propose an Iterative utiliTy judgm fraEntMework (ITEM) to promote each step in Retrieval-Augmented Generation (RAG)<n>RAG's three core components -- relevance ranking derived from retrieval models, utility judgments, and answer generation -- align with Schutz's philosophical system of relevances.<n> Experimental results demonstrate significant improvements of ITEM in utility judgments, ranking, and answer generation upon representative baselines.
  arXiv  Detail & Related papers  (2024-06-17T07:52:42Z)
• Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs [58.620269228776294]
  We propose a task-agnostic framework for resolving ambiguity by asking users clarifying questions. We evaluate systems across three NLP applications: question answering, machine translation and natural language inference. We find that intent-sim is robust, demonstrating improvements across a wide range of NLP tasks and LMs.
  arXiv  Detail & Related papers  (2023-11-16T00:18:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.