Trust by Design: Trust Calibration Through Non-Advisory Socratic Dialogue in Conversational Agents

• URL: http://arxiv.org/abs/2609.14818v1
• Date: Sun, 13 Sep 2026 22:19:18 GMT
• Title: Trust by Design: Trust Calibration Through Non-Advisory Socratic Dialogue in Conversational Agents
• Abstract summary: We present CASELy, a conversational agent designed to limit its own authority through non-advisory Socratic dialogue.<n>The agent asks reflective questions grounded exclusively in user input and refuses to provide advice, recommendations, or interpretations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As conversational AI systems increasingly operate in sensitive domains, the central challenge shifts from usability to trust calibration, ensuring that users rely on systems neither too much nor too little. Systems that provide advice or interpretations risk encouraging inappropriate reliance, particularly when users perceive AI outputs as authoritative. We present CASELy, a conversational agent explicitly designed to limit its own authority through non-advisory Socratic dialogue. The agent asks reflective questions grounded exclusively in user input and refuses to provide advice, recommendations, or interpretations. This design operationalizes trust calibration by constraining agent agency rather than optimizing capability. In a pilot randomized controlled study with higher education students, participants interacting with the Socratic dialogue reported substantially higher user experience (UEQ-S overall = 1.50) compared to a non-dialogue control (0). Qualitative findings identify three mechanisms supporting calibrated trust: transparency through visible grounding, preservation of user decision authority, and reduced fear of judgment. We argue that appropriate reliance can be achieved through interactional constraints, offering a design pattern for trustworthy conversational AI in sensitive contexts.

Related papers

• Personalized to Persuade: The Effects of Contextualization and Warmth on Trust and Reliance in Conversational AI [8.544772506500188]
  Personalization has been identified as a persuasive strategy in politics or in marketing.<n>We examine how contextualization, combined with conversational warmth, shapes reliance and persuasiveness of an AI assistant arguing against expert recommendations.
  arXiv  Detail & Related papers  (2026-05-29T13:07:03Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• Co-FactChecker: A Framework for Human-AI Collaborative Claim Verification Using Large Reasoning Models [99.26398772227684]
  We propose Co-FactChecker, a framework for human-AI collaborative claim verification.<n>Co-FactChecker translates expert feedback into trace-edits that introduce targeted modifications to the trace.<n>Human evaluations show that Co-FactChecker is preferred over multi-turn dialogue.
  arXiv  Detail & Related papers  (2026-04-15T10:35:00Z)
• When Contextual Inference Fails: Cancelability in Interactive Instruction Following [51.2195840589474]
  We introduce Build What I Mean, an interactive benchmark for contextual meaning construction.<n>In BWIM, models must resolve ambiguity by either performing a contextual inference or requesting clarification at a small communication cost.<n>We observe suboptimal strategies, such as partner-blind over-clarification and question-averse guessing under uncertainty.
  arXiv  Detail & Related papers  (2026-03-20T14:46:59Z)
• Dynamic Trust Calibration Using Contextual Bandits [1.8563342761346613]
  Excessive trust can lead users to accept AI-generated outputs without question.<n>Insufficient trust may result in disregarding valuable insights from AI systems.<n>There is currently no definitive and objective method for measuring trust calibration between humans and AI.
  arXiv  Detail & Related papers  (2025-09-27T21:06:17Z)
• "Even explanations will not help in trusting [this] fundamentally biased system": A Predictive Policing Case-Study [8.240854389254222]
  The use of AI systems in high-risk domains have often led users to either under-trust it, potentially causing inadequate reliance or over-trust it, resulting in over-compliance.<n>Past research has indicated that explanations provided by AI systems can enhance user understanding of when to trust or not trust the system.<n>This study explores the impact of different explanation types and user expertise on establishing appropriate trust in AI-based predictive policing.
  arXiv  Detail & Related papers  (2025-04-15T09:43:48Z)
• DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metrics [52.242449026151846]
  Multi-agent debates have been introduced to improve the accuracy of Large Language Models (LLMs)<n>We propose DebUnc, a debate framework that uses uncertainty metrics to assess agent confidence.
  arXiv  Detail & Related papers  (2024-07-08T22:15:01Z)
• A Diachronic Perspective on User Trust in AI under Uncertainty [52.44939679369428]
  Modern NLP systems are often uncalibrated, resulting in confidently incorrect predictions that undermine user trust. We study the evolution of user trust in response to trust-eroding events using a betting game.
  arXiv  Detail & Related papers  (2023-10-20T14:41:46Z)
• A Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspective [0.0]
  User trust in Artificial Intelligence (AI) enabled systems has been increasingly recognized and proven as a key element to fostering adoption. This review aims to provide an overview of the user trust definitions, influencing factors, and measurement methods from 23 empirical studies.
  arXiv  Detail & Related papers  (2023-04-18T07:58:09Z)
• Designing for Responsible Trust in AI Systems: A Communication Perspective [56.80107647520364]
  We draw from communication theories and literature on trust in technologies to develop a conceptual model called MATCH. We highlight transparency and interaction as AI systems' affordances that present a wide range of trustworthiness cues to users. We propose a checklist of requirements to help technology creators identify appropriate cues to use.
  arXiv  Detail & Related papers  (2022-04-29T00:14:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.