Epistemic Trustworthiness in Generative AI: A Normative Framework for Warranted Reliance in High-Stakes Workflows

• URL: http://arxiv.org/abs/2608.05602v2
• Date: Sun, 09 Aug 2026 22:23:05 GMT
• Title: Epistemic Trustworthiness in Generative AI: A Normative Framework for Warranted Reliance in High-Stakes Workflows
• Abstract summary: Generative AI systems are increasingly deployed in high-stakes professional contexts.<n>Existing frameworks ask whether AI outputs are accurate, fair, explainable, safe, or trusted by users.<n>We argue that this requires an account of trustworthiness: what makes a system worthy of reliance.
• Score: 20.544778365691503
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI systems are increasingly deployed in high-stakes professional contexts, where their outputs shape what users believe, how they reason, and what they treat as settled. This raises a central question for responsible AI: under what conditions is reliance on generative AI outputs epistemically warranted rather than behaviourally induced? Existing frameworks largely ask whether AI outputs are accurate, fair, explainable, safe, or trusted by users. These questions remain necessary, and each can contribute to warranted reliance. However, they do not directly specify warranted reliance as a distinct evaluative target: the conditions under which users are justified in treating AI outputs as inputs into their own reasoning. We argue that this requires an account of epistemic trustworthiness: what makes a system epistemically worthy of reliance. Drawing on philosophical accounts of trustworthiness as competence and audience-orientation, we develop a constitutive normative framework comprising three jointly necessary and non-fungible conditions. First, epistemic humility requires systems to represent and communicate the limits of their competence. Second, epistemic access requires systems to enable users to inspect, question, and contest outputs in context. Third, resistance to epistemic injustice requires systems to recognise users as legitimate epistemic agents and avoid marginalising their knowledge and experience. Through real-world case analyses in legal reasoning, medical reasoning, and hiring, we show how failures of epistemic humility, epistemic access, and resistance to epistemic injustice can produce consequential harms that standard measures of accuracy, fairness, and usability do not address on their own. We conclude by outlining design and evaluation implications for GenAI systems organised around epistemically warranted reliance rather than output correctness alone.

Related papers

• Validity, Reliability, and Transparency in Artificial Intelligence Regulation [2.588958866512272]
  Article argues that modern AI raises concerns of construct validity, confounding, representativeness, distribution shift, and fairness trade-offs.<n>We argue that inference should function as a precondition for proportionality assessment and deployment approval.
  arXiv  Detail & Related papers  (2026-08-06T09:36:15Z)
• Trustworthy AI Suffers from Invariance Conflicts and Causality is The Solution [80.98492754957466]
  Core trustworthy AI objectives, such as fairness, robustness, privacy, and explainability, are hard to achieve simultaneously.<n>This paper argues that causality is necessary to understand and balance trade-offs in performance and multiple objectives of trustworthy AI.
  arXiv  Detail & Related papers  (2026-05-04T14:26:28Z)
• AI Integrity: A New Paradigm for Verifiable AI Governance [3.7184769644515896]
  AI systems increasingly shape high-stakes decisions in healthcare, law, defense, and education.<n>Existing governance paradigms -- AI Ethics, AI Safety, and AI Alignment -- share a common limitation: they evaluate outcomes rather than verifying the reasoning process itself.<n>This paper introduces AI Integrity, a concept defined as a state in which the Authority Stack of an AI system is protected from corruption, contamination, manipulation, and bias.
  arXiv  Detail & Related papers  (2026-04-13T06:45:30Z)
• Why Avoid Generative Legal AI Systems? Hallucination, Overreliance, and their Impact on Explainability [0.0]
  This article argues that the deployment of generative AI systems in legal profession requires strong restraint due to the critical risks of hallucination and overreliance.<n>Central to this analysis is the definition of Generative Legal AI (GLAI), an umbrella term for systems specifically adapted for the legal domain.
  arXiv  Detail & Related papers  (2026-03-16T21:40:43Z)
• Architecting Trust in Artificial Epistemic Agents [6.054175970982164]
  We argue that the potential impact of AI agents on practices of knowledge creation, curation and synthesis requires a shift in evaluation and governance of AI.<n>We propose a framework centered on building and cultivating the trustworthiness of epistemic AI agents.<n>This normative roadmap provides a path toward ensuring that future AI systems act as reliable partners in a robust and inclusive knowledge ecosystem.
  arXiv  Detail & Related papers  (2026-03-03T13:12:33Z)
• AI Deception: Risks, Dynamics, and Controls [153.71048309527225]
  This project provides a comprehensive and up-to-date overview of the AI deception field.<n>We identify a formal definition of AI deception, grounded in signaling theory from studies of animal deception.<n>We organize the landscape of AI deception research as a deception cycle, consisting of two key components: deception emergence and deception treatment.
  arXiv  Detail & Related papers  (2025-11-27T16:56:04Z)
• Epistemic Deference to AI [0.01692139688032578]
  I argue that some AI systems are Artificial Epistemic Authorities (AEAs)<n>AEAs should function as contributory reasons rather than outright replacements for a user's independent epistemic considerations.<n>While demanding in practice, this account offers a principled way to determine when AI deference is justified.
  arXiv  Detail & Related papers  (2025-10-23T22:55:51Z)
• Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor [83.99510317617694]
  We argue that a broader conception of what rigorous AI research and practice should entail is needed.<n>We aim to provide useful language and a framework for much-needed dialogue about the AI community's work.
  arXiv  Detail & Related papers  (2025-06-17T15:44:41Z)
• Causal Responsibility Attribution for Human-AI Collaboration [62.474732677086855]
  This paper presents a causal framework using Structural Causal Models (SCMs) to systematically attribute responsibility in human-AI systems. Two case studies illustrate the framework's adaptability in diverse human-AI collaboration scenarios.
  arXiv  Detail & Related papers  (2024-11-05T17:17:45Z)
• Engineering Trustworthy AI: A Developer Guide for Empirical Risk Minimization [53.80919781981027]
  Key requirements for trustworthy AI can be translated into design choices for the components of empirical risk minimization. We hope to provide actionable guidance for building AI systems that meet emerging standards for trustworthiness of AI.
  arXiv  Detail & Related papers  (2024-10-25T07:53:32Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.