The Imbalanced User-AI Relationships as an Ethical Failure of Front-End Design in Healthcare AI

• URL: http://arxiv.org/abs/2604.22767v1
• Date: Tue, 24 Mar 2026 17:15:57 GMT
• Title: The Imbalanced User-AI Relationships as an Ethical Failure of Front-End Design in Healthcare AI
• Abstract summary: We show how patients are rendered highly visible to AI systems through data inference, yet cannot understand, question or influence how they are represented.<n>We propose interventions for more balanced, participatory user-AI relationships in healthcare.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ethical discourse on AI in healthcare has focused predominantly on back-end concerns such as bias, fairness and explainability, while the front-end interface, where patients and clinicians actually encounter AI outputs, remains under explored. This paper identifies imbalanced user-AI relationships as a distinct class of front-end ethical failure: patients are rendered highly visible to AI systems through data inference, yet cannot understand, question or influence how they are represented. Through the concept of asymmetric legibility and a chat-based telemedicine case, we show how design choices e.g., default recommendations, restricted inputs and suppressed uncertainty, undermine agency, clinician judgment and human oversight even where systems are technically accurate. We propose reciprocity as a design orientation and offer interventions for more balanced, participatory user-AI relationships in healthcare.

Related papers

• When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and Adoption [45.10829096284761]
  Complex AI systems can appear highly accurate while making costly errors or embedding hidden defects.<n>This paper provides the first experimental evidence on the role of information asymmetries and disclosure designs in shaping user adoption of AI systems.
  arXiv  Detail & Related papers  (2026-01-29T12:49:28Z)
• Epistemology gives a Future to Complementarity in Human-AI Interactions [42.371764229953165]
  complementarity is the claim that a human supported by an AI system can outperform either alone in a decision-making process.<n>We argue that historical instances of complementarity function as evidence that a given human-AI interaction is a reliable process.
  arXiv  Detail & Related papers  (2026-01-14T21:04:28Z)
• Explainable AI as a Double-Edged Sword in Dermatology: The Impact on Clinicians versus The Public [46.86429592892395]
  explainable AI (XAI) addresses this by providing AI decision-making insight.<n>We present results from two large-scale experiments combining a fairness-based diagnosis AI model and different XAI explanations.
  arXiv  Detail & Related papers  (2025-12-14T00:06:06Z)
• Echoes of AI Harms: A Human-LLM Synergistic Framework for Bias-Driven Harm Anticipation [1.5892420496333068]
  ECHO is a novel framework for proactive AI harm anticipation.<n>It enables early-stage detection of bias-to-harm pathways.<n>We validate ECHO in two high-stakes domains (disease diagnosis and hiring)
  arXiv  Detail & Related papers  (2025-11-27T07:25:21Z)
• A quality of mercy is not trained: the imagined vs. the practiced in healthcare process-specialized AI development [0.0]
  We show how early representational decisions narrowed what the AI could support, resulting in the premature exclusion of key ethical dimensions from system design.<n>Our findings surface the moral consequences of abstraction and call for a more situated approach to designing healthcare process-specialized artificial intelligence systems.
  arXiv  Detail & Related papers  (2025-10-22T14:48:35Z)
• A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoption [6.7236795813629]
  We propose a design framework to support developers in embedding Trustworthy AI principles into medical AI systems.<n>We focus on cardiovascular diseases, a field marked by both high prevalence and active AI innovation.
  arXiv  Detail & Related papers  (2025-04-27T09:57:35Z)
• Ethical Challenges and Evolving Strategies in the Integration of Artificial Intelligence into Clinical Practice [1.0301404234578682]
  We focus on five critical ethical concerns: justice and fairness, transparency, patient consent and confidentiality, accountability, and patient-centered and equitable care.<n>The paper explores how bias, lack of transparency, and challenges in maintaining patient trust can undermine the effectiveness and fairness of AI applications in healthcare.
  arXiv  Detail & Related papers  (2024-11-18T00:52:22Z)
• Which Client is Reliable?: A Reliable and Personalized Prompt-based Federated Learning for Medical Image Question Answering [51.26412822853409]
  We present a novel personalized federated learning (pFL) method for medical visual question answering (VQA) models. Our method introduces learnable prompts into a Transformer architecture to efficiently train it on diverse medical datasets without massive computational costs.
  arXiv  Detail & Related papers  (2024-10-23T00:31:17Z)
• The doctor will polygraph you now: ethical concerns with AI for fact-checking patients [0.23248585800296404]
  Artificial intelligence (AI) methods have been proposed for the prediction of social behaviors. This raises novel ethical concerns about respect, privacy, and control over patient data.
  arXiv  Detail & Related papers  (2024-08-15T02:55:30Z)
• What Do End-Users Really Want? Investigation of Human-Centered XAI for Mobile Health Apps [69.53730499849023]
  We present a user-centered persona concept to evaluate explainable AI (XAI) Results show that users' demographics and personality, as well as the type of explanation, impact explanation preferences. Our insights bring an interactive, human-centered XAI closer to practical application.
  arXiv  Detail & Related papers  (2022-10-07T12:51:27Z)
• Towards the Use of Saliency Maps for Explaining Low-Quality Electrocardiograms to End Users [51.644376281196394]
  When using medical images for diagnosis, it is important that the images are of high quality.<n>In telemedicine, a common problem is that the quality issue is only flagged once the patient has left the clinic, meaning they must return in order to have the exam redone.<n>This paper reports on the development of an AI system for flagging and explaining low-quality medical images in real-time.
  arXiv  Detail & Related papers  (2022-07-06T14:53:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.