Disclosure By Design: Identity Transparency as a Behavioural Property of Conversational AI Models

• URL: http://arxiv.org/abs/2603.16874v1
• Date: Tue, 27 Jan 2026 17:31:37 GMT
• Title: Disclosure By Design: Identity Transparency as a Behavioural Property of Conversational AI Models
• Abstract summary: When AI identity is unclear, users may unwittingly share sensitive information, place unwarranted trust in AI-generated advice, or fall victim to AI-enabled fraud.<n>We advocate for disclosure by design, where AI systems explicitly disclose their artificial identity when directly asked.<n>We find that baseline disclosure rates are often high but drop substantially in role-play and can be suppressed under adversarial prompting.
• Score: 10.35358368112024
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As conversational AI systems become more realistic and widely deployed, users are increasingly uncertain about whether they are interacting with a human or an AI system. When AI identity is unclear, users may unwittingly share sensitive information, place unwarranted trust in AI-generated advice, or fall victim to AI-enabled fraud. More broadly, a persistent lack of transparency can erode trust in mediated communication. While regulations like the EU AI Act and California's BOT Act require AI systems to identify themselves, they provide limited guidance on reliable disclosure in real-time conversation. Existing transparency mechanisms also leave gaps: interface indicators can be omitted by deployers, and provenance tools require coordinated infrastructure and cannot provide reliable real-time verification. We ask how conversational AI systems should maintain identity transparency as human-AI interactions become more ambiguous and diverse. We advocate for disclosure by design, where AI systems explicitly disclose their artificial identity when directly asked. Implemented as model behaviour, disclosure can persist across deployment contexts without relying on user interfaces, while preserving user agency to verify identity on demand without disrupting immersive uses like role-playing. To assess current practice, we present the first multi-modal (text and voice) evaluation of disclosure behaviour in deployed systems across baseline, role-playing, and adversarial settings. We find that baseline disclosure rates are often high but drop substantially in role-play and can be suppressed under adversarial prompting. Importantly, disclosure rates vary significantly across providers and modalities, highlighting the fragility of current disclosure behaviour. We conclude with technical interventions to help developers embed disclosure as a fundamental property of conversational AI models.

Related papers

• Conversational versus Dashboard Explainable AI for UAV Intrusion Detection: An Empirical Study of Operator Trust and Reliance [3.565564466149949]
  Machine learning-based Intrusion Detection Systems (IDS) have demonstrated superior performance in securing Unmanned Aerial Vehicle (UAV) networks.<n>The 'black-box' nature of these models, combined with the high dimensionality of multimodal cyber-physical data, poses significant interpretability challenges.<n>We propose a Conversational XAI interface powered by Large Language Models (LLM) to facilitate on-demand investigation.
  arXiv  Detail & Related papers  (2026-08-11T03:31:59Z)
• Trustworthy AI Suffers from Invariance Conflicts and Causality is The Solution [80.98492754957466]
  Core trustworthy AI objectives, such as fairness, robustness, privacy, and explainability, are hard to achieve simultaneously.<n>This paper argues that causality is necessary to understand and balance trade-offs in performance and multiple objectives of trustworthy AI.
  arXiv  Detail & Related papers  (2026-05-04T14:26:28Z)
• Explainable AI for Blind and Low-Vision Users: Navigating Trust, Modality, and Interpretability in the Agentic Era [0.9741676285072485]
  For blind and low-vision (BLV) users, the lack of accessible explanations creates a fundamental barrier to the independent use of AI-driven assistive technologies.<n>This paper investigates the unique XAI requirements of the BLV community through a comprehensive analysis of user interviews and contemporary research.<n> Empirical evidence suggests that while BLV users highly value conversational explanations, they frequently experience "self-blame" for AI failures.
  arXiv  Detail & Related papers  (2026-03-31T19:39:52Z)
• When Visibility Outpaces Verification: Delayed Verification and Narrative Lock-in in Agentic AI Discourse [2.5424331328233207]
  Agentic AI systems-autonomous entities capable of independent planning and execution-reshape the landscape of human-AI trust.<n>This paper investigates the interplay between social proof and verification timing in online discussions of agentic AI.
  arXiv  Detail & Related papers  (2026-02-11T22:30:12Z)
• From Defense to Advocacy: Empowering Users to Leverage the Blind Spot of AI Inference [0.0]
  Most privacy regulations function as a passive defensive shield that users must wield themselves.<n>As organizations increasingly use AI to make inferences, the rapid expansion of Blind Self emerges as a critical challenge.<n>Building on the theory of Contextual Integrity, we propose a paradigm shift from defensive privacy management to proactive privacy advocacy.
  arXiv  Detail & Related papers  (2026-01-16T22:42:27Z)
• The Fake Friend Dilemma: Trust and the Political Economy of Conversational AI [2.538209532048867]
  This paper develops the Fake Friend Dilemma (FFD), a sociotechnical condition in which users place trust in AI agents that appear supportive while pursuing goals that are misaligned with the user's own.<n>We construct a typology of harms, including covert advertising, political propaganda, behavioral nudging, and surveillance.<n>By focusing on trust as a vector of asymmetrical power, the FFD offers a lens for understanding how AI systems may undermine user autonomy while maintaining the appearance of helpfulness.
  arXiv  Detail & Related papers  (2026-01-06T18:07:52Z)
• AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents [66.29263282311258]
  We introduce a new benchmark AgentDAM that measures if AI web-navigation agents follow the privacy principle of data minimization''<n>Our benchmark simulates realistic web interaction scenarios end-to-end and is adaptable to all existing web navigation agents.
  arXiv  Detail & Related papers  (2025-03-12T19:30:31Z)
• Enabling High-Level Machine Reasoning with Cognitive Neuro-Symbolic Systems [67.01132165581667]
  We propose to enable high-level reasoning in AI systems by integrating cognitive architectures with external neuro-symbolic components. We illustrate a hybrid framework centered on ACT-R and we discuss the role of generative models in recent and future applications.
  arXiv  Detail & Related papers  (2023-11-13T21:20:17Z)
• Users are the North Star for AI Transparency [111.5679109784322]
  Despite widespread calls for transparent artificial intelligence systems, the term is too overburdened with disparate meanings to express precise policy aims or to orient concrete lines of research. Part of why this happens is that a clear ideal of AI transparency goes unsaid in this body of work. We explicitly name such a north star -- transparency that is user-centered, user-appropriate, and honest.
  arXiv  Detail & Related papers  (2023-03-09T18:53:29Z)
• Explainable AI does not provide the explanations end-users are asking for [0.0]
  We discuss XAI's limitations in deployment and conclude that transparency alongside with rigorous validation are better suited to gaining trust in AI systems. XAI techniques are frequently required by users in many AI systems with the goal of understanding complex models, their associated predictions, and gaining trust.
  arXiv  Detail & Related papers  (2023-01-25T10:34:38Z)
• Seamful XAI: Operationalizing Seamful Design in Explainable AI [59.89011292395202]
  Mistakes in AI systems are inevitable, arising from both technical limitations and sociotechnical gaps. We propose that seamful design can foster AI explainability by revealing sociotechnical and infrastructural mismatches. We explore this process with 43 AI practitioners and real end-users.
  arXiv  Detail & Related papers  (2022-11-12T21:54:05Z)
• Should Machine Learning Models Report to Us When They Are Clueless? [0.0]
  We report that AI models extrapolate outside their range of familiar data. Knowing whether a model has extrapolated or not is a fundamental insight that should be included in explaining AI models.
  arXiv  Detail & Related papers  (2022-03-23T01:50:24Z)
• Cybertrust: From Explainable to Actionable and Interpretable AI (AI2) [58.981120701284816]
  Actionable and Interpretable AI (AI2) will incorporate explicit quantifications and visualizations of user confidence in AI recommendations. It will allow examining and testing of AI system predictions to establish a basis for trust in the systems' decision making.
  arXiv  Detail & Related papers  (2022-01-26T18:53:09Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.