Linguistic Comparison of AI- and Human-Written Responses to Online Mental Health Queries

• URL: http://arxiv.org/abs/2504.09271v1
• Date: Sat, 12 Apr 2025 16:20:02 GMT
• Title: Linguistic Comparison of AI- and Human-Written Responses to Online Mental Health Queries
• Authors: Koustuv Saha, Yoshee Jain, Munmun De Choudhury,
• Abstract summary: generative AI and large language models (LLMs) have introduced new possibilities for scalable, around-the-clock mental health assistance.<n>In this study, we harnessed 24,114 posts and 138,758 online community (OC) responses from 55 online support communities on Reddit.<n>Our findings revealed that AI responses are more verbose, readable, and analytically structured, but lack linguistic diversity and personal narratives inherent in human-human interactions.
• Score: 17.133059430285627
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ubiquity and widespread use of digital and online technologies have transformed mental health support, with online mental health communities (OMHCs) providing safe spaces for peer support. More recently, generative AI and large language models (LLMs) have introduced new possibilities for scalable, around-the-clock mental health assistance that could potentially augment and supplement the capabilities of OMHCs. Although genAI shows promise in delivering immediate and personalized responses, their effectiveness in replicating the nuanced, experience-based support of human peers remains an open question. In this study, we harnessed 24,114 posts and 138,758 online community (OC) responses from 55 OMHCs on Reddit. We prompted several state-of-the-art LLMs (GPT-4-Turbo, Llama-3, and Mistral-7B) with these posts, and compared their (AI) responses to human-written (OC) responses based on a variety of linguistic measures across psycholinguistics and lexico-semantics. Our findings revealed that AI responses are more verbose, readable, and analytically structured, but lack linguistic diversity and personal narratives inherent in human-human interactions. Through a qualitative examination, we found validation as well as complementary insights into the nature of AI responses, such as its neutrality of stance and the absence of seeking back-and-forth clarifications. We discuss the ethical and practical implications of integrating generative AI into OMHCs, advocating for frameworks that balance AI's scalability and timeliness with the irreplaceable authenticity, social interactiveness, and expertise of human connections that form the ethos of online support communities.

Related papers

• Measurement of LLM's Philosophies of Human Nature [113.47929131143766]
  We design the standardized psychological scale specifically targeting large language models (LLM)<n>We show that current LLMs exhibit a systemic lack of trust in humans.<n>We propose a mental loop learning framework, which enables LLM to continuously optimize its value system.
  arXiv  Detail & Related papers  (2025-04-03T06:22:19Z)
• Envisioning an AI-Enhanced Mental Health Ecosystem [1.534667887016089]
  We explore various AI applications in peer support, self-help interventions, proactive monitoring, and data-driven insights.<n>We propose a hybrid ecosystem where AI assists but does not replace human providers, emphasising responsible deployment and evaluation.
  arXiv  Detail & Related papers  (2025-03-19T04:21:38Z)
• Emergence of human-like polarization among large language model agents [61.622596148368906]
  We simulate a networked system involving thousands of large language model agents, discovering their social interactions, result in human-like polarization.<n>Similarities between humans and LLM agents raise concerns about their capacity to amplify societal polarization, but also hold the potential to serve as a valuable testbed for identifying plausible strategies to mitigate it.
  arXiv  Detail & Related papers  (2025-01-09T11:45:05Z)
• Aligning Generalisation Between Humans and Machines [74.120848518198]
  Recent advances in AI have resulted in technology that can support humans in scientific discovery and decision support but may also disrupt democracies and target individuals. The responsible use of AI increasingly shows the need for human-AI teaming. A crucial yet often overlooked aspect of these interactions is the different ways in which humans and machines generalise.
  arXiv  Detail & Related papers  (2024-11-23T18:36:07Z)
• How Performance Pressure Influences AI-Assisted Decision Making [57.53469908423318]
  We show how pressure and explainable AI (XAI) techniques interact with AI advice-taking behavior.<n>Our results show complex interaction effects, with different combinations of pressure and XAI techniques either improving or worsening AI advice taking behavior.
  arXiv  Detail & Related papers  (2024-10-21T22:39:52Z)
• Enhancing Mental Health Support through Human-AI Collaboration: Toward Secure and Empathetic AI-enabled chatbots [0.0]
  This paper explores the potential of AI-enabled chatbots as a scalable solution. We assess their ability to deliver empathetic, meaningful responses in mental health contexts. We propose a federated learning framework that ensures data privacy, reduces bias, and integrates continuous validation from clinicians to enhance response quality.
  arXiv  Detail & Related papers  (2024-09-17T20:49:13Z)
• Empathy Through Multimodality in Conversational Interfaces [1.360649555639909]
  Conversational Health Agents (CHAs) are redefining healthcare by offering nuanced support that transcends textual analysis to incorporate emotional intelligence. This paper introduces an LLM-based CHA engineered for rich, multimodal dialogue-especially in the realm of mental health support. It adeptly interprets and responds to users' emotional states by analyzing multimodal cues, thus delivering contextually aware and empathetically resonant verbal responses.
  arXiv  Detail & Related papers  (2024-05-08T02:48:29Z)
• Challenges of Large Language Models for Mental Health Counseling [4.604003661048267]
  The global mental health crisis is looming with a rapid increase in mental disorders, limited resources, and the social stigma of seeking treatment. The application of large language models (LLMs) in the mental health domain raises concerns regarding the accuracy, effectiveness, and reliability of the information provided. This paper investigates the major challenges associated with the development of LLMs for psychological counseling, including model hallucination, interpretability, bias, privacy, and clinical effectiveness.
  arXiv  Detail & Related papers  (2023-11-23T08:56:41Z)
• Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision [84.31474052176343]
  Recent AI-assistant agents, such as ChatGPT, rely on supervised fine-tuning (SFT) with human annotations and reinforcement learning from human feedback to align the output with human intentions. This dependence can significantly constrain the true potential of AI-assistant agents due to the high cost of obtaining human supervision. We propose a novel approach called SELF-ALIGN, which combines principle-driven reasoning and the generative power of LLMs for the self-alignment of AI agents with minimal human supervision.
  arXiv  Detail & Related papers  (2023-05-04T17:59:28Z)
• The Role of AI in Drug Discovery: Challenges, Opportunities, and Strategies [97.5153823429076]
  The benefits, challenges and drawbacks of AI in this field are reviewed. The use of data augmentation, explainable AI, and the integration of AI with traditional experimental methods are also discussed.
  arXiv  Detail & Related papers  (2022-12-08T23:23:39Z)
• Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Support [10.743204843534512]
  We develop Hailey, an AI-in-the-loop agent that provides just-in-time feedback to help participants who provide support (peer supporters) respond more empathically to those seeking help (support seekers) We show that our Human-AI collaboration approach leads to a 19.60% increase in conversational empathy between peers overall. We find a larger 38.88% increase in empathy within the subsample of peer supporters who self-identify as experiencing difficulty providing support.
  arXiv  Detail & Related papers  (2022-03-28T23:37:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.