Hide or Highlight: Understanding the Impact of Factuality Expression on User Trust

• URL: http://arxiv.org/abs/2508.07095v1
• Date: Sat, 09 Aug 2025 20:45:21 GMT
• Title: Hide or Highlight: Understanding the Impact of Factuality Expression on User Trust
• Authors: Hyo Jin Do, Werner Geyer,
• Abstract summary: We tested four different ways of disclosing an AI-generated output with factuality assessments.<n>We found that the opaque and ambiguity strategies led to higher trust while maintaining perceived answer quality.
• Score: 1.2478643689100954
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large language models are known to produce outputs that are plausible but factually incorrect. To prevent people from making erroneous decisions by blindly trusting AI, researchers have explored various ways of communicating factuality estimates in AI-generated outputs to end-users. However, little is known about whether revealing content estimated to be factually incorrect influences users' trust when compared to hiding it altogether. We tested four different ways of disclosing an AI-generated output with factuality assessments: transparent (highlights less factual content), attention (highlights factual content), opaque (removes less factual content), ambiguity (makes less factual content vague), and compared them with a baseline response without factuality information. We conducted a human subjects research (N = 148) using the strategies in question-answering scenarios. We found that the opaque and ambiguity strategies led to higher trust while maintaining perceived answer quality, compared to the other strategies. We discuss the efficacy of hiding presumably less factual content to build end-user trust.

Related papers

• When AI Persuades: Adversarial Explanation Attacks on Human Trust in AI-Assisted Decision Making [7.170587130743388]
  Large Language Models generate fluent natural-language explanations that shape how users perceive and trust AI outputs.<n>We introduce adversarial explanation attacks (AEAs), where an attacker manipulates the framing of LLM-generated explanations to modulate human trust in incorrect outputs.<n>This is the first systematic security study that treats explanations as an adversarial cognitive channel and quantifies their impact on human trust in AI-assisted decision making.
  arXiv  Detail & Related papers  (2026-02-03T20:42:44Z)
• Full Disclosure, Less Trust? How the Level of Detail about AI Use in News Writing Affects Readers' Trust [10.22272389430846]
  Findings show that not all AI disclosures lead to a transparency dilemma, but instead reflect a trade-off between readers' desire for more transparency and their trust in AI-assisted news content.
  arXiv  Detail & Related papers  (2026-01-14T16:45:45Z)
• Evaluating Language Model Reasoning about Confidential Information [95.64687778185703]
  We study whether language models exhibit contextual robustness, or the capability to adhere to context-dependent safety specifications.<n>We develop a benchmark (PasswordEval) that measures whether language models can correctly determine when a user request is authorized.<n>We find that current open- and closed-source models struggle with this seemingly simple task, and that, perhaps surprisingly, reasoning capabilities do not generally improve performance.
  arXiv  Detail & Related papers  (2025-08-27T15:39:46Z)
• Superhuman Game AI Disclosure: Expertise and Context Moderate Effects on Trust and Fairness [13.63944785085617]
  We investigate how capability disclosure influenced behaviors with a superhuman game AI in competitive StarCraft II scenarios.<n>Our results reveal transparency is double-edged: while disclosure could alleviate suspicion, it also provoked frustration and strategic defeatism.<n>This work demonstrates that transparency is not a cure-all; successfully leveraging disclosure to enhance trust and accountability requires careful tailoring to user characteristics.
  arXiv  Detail & Related papers  (2025-01-31T05:50:50Z)
• "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust [51.542856739181474]
  We show how different natural language expressions of uncertainty impact participants' reliance, trust, and overall task performance. We find that first-person expressions decrease participants' confidence in the system and tendency to agree with the system's answers, while increasing participants' accuracy. Our findings suggest that using natural language expressions of uncertainty may be an effective approach for reducing overreliance on LLMs, but that the precise language used matters.
  arXiv  Detail & Related papers  (2024-05-01T16:43:55Z)
• Do You Trust ChatGPT? -- Perceived Credibility of Human and AI-Generated Content [0.8602553195689513]
  This paper examines how individuals perceive the credibility of content originating from human authors versus content generated by large language models. Surprisingly, our results demonstrate that regardless of the user interface presentation, participants tend to attribute similar levels of credibility. Participants also do not report any different perceptions of competence and trustworthiness between human and AI-generated content.
  arXiv  Detail & Related papers  (2023-09-05T18:29:29Z)
• Fact-checking information from large language models can decrease headline discernment [6.814801748069122]
  We investigate the impact of fact-checking information generated by a popular large language model on belief in, and sharing intent of, political news headlines. We find that this information does not significantly improve participants' ability to discern headline accuracy or share accurate news. Our findings highlight an important source of potential harm stemming from AI applications.
  arXiv  Detail & Related papers  (2023-08-21T15:47:37Z)
• Survey of Trustworthy AI: A Meta Decision of AI [0.41292255339309647]
  Trusting an opaque system involves deciding on the level of Trustworthy AI (TAI) To underpin these domains, we create ten dimensions to measure trust: explainability/transparency, fairness/diversity, generalizability, privacy, data governance, safety/robustness, accountability, reliability, and sustainability.
  arXiv  Detail & Related papers  (2023-06-01T06:25:01Z)
• Context-faithful Prompting for Large Language Models [51.194410884263135]
  Large language models (LLMs) encode parametric knowledge about world facts. Their reliance on parametric knowledge may cause them to overlook contextual cues, leading to incorrect predictions in context-sensitive NLP tasks. We assess and enhance LLMs' contextual faithfulness in two aspects: knowledge conflict and prediction with abstention.
  arXiv  Detail & Related papers  (2023-03-20T17:54:58Z)
• The Who in XAI: How AI Background Shapes Perceptions of AI Explanations [61.49776160925216]
  We conduct a mixed-methods study of how two different groups--people with and without AI background--perceive different types of AI explanations. We find that (1) both groups showed unwarranted faith in numbers for different reasons and (2) each group found value in different explanations beyond their intended design.
  arXiv  Detail & Related papers  (2021-07-28T17:32:04Z)
• Machine Learning Explanations to Prevent Overtrust in Fake News Detection [64.46876057393703]
  This research investigates the effects of an Explainable AI assistant embedded in news review platforms for combating the propagation of fake news. We design a news reviewing and sharing interface, create a dataset of news stories, and train four interpretable fake news detection algorithms. For a deeper understanding of Explainable AI systems, we discuss interactions between user engagement, mental model, trust, and performance measures in the process of explaining.
  arXiv  Detail & Related papers  (2020-07-24T05:42:29Z)
• Information Consumption and Social Response in a Segregated Environment: the Case of Gab [74.5095691235917]
  This work provides a characterization of the interaction patterns within Gab around the COVID-19 topic. We find that there are no strong statistical differences in the social response to questionable and reliable content. Our results provide insights toward the understanding of coordinated inauthentic behavior and on the early-warning of information operation.
  arXiv  Detail & Related papers  (2020-06-03T11:34:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.