LLMs in social services: How does chatbot accuracy affect human accuracy?

• URL: http://arxiv.org/abs/2603.11213v1
• Date: Wed, 11 Mar 2026 18:28:52 GMT
• Title: LLMs in social services: How does chatbot accuracy affect human accuracy?
• Abstract summary: We measure the potential effects of chatbots on caseworkers' ability to provide accurate guidance.<n>We conducted a randomized experiment with caseworkers recruited from nonprofit outreach organizations in Los Angeles.<n>High-quality chatbots (96-100% accurate) improved caseworker accuracy by 27 percentage points.
• Score: 4.837450441799026
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Social service programs like the Supplemental Nutrition Assistance Program (SNAP, or food stamps) have eligibility rules that can be challenging to understand. For nonprofit caseworkers who often support clients in navigating a dozen or more complex programs, LLM-based chatbots may offer a means to provide better, faster help to clients whose situations may be less common. In this paper, we measure the potential effects of LLM-based chatbot suggestions on caseworkers' ability to provide accurate guidance. We first created a 770-question multiple-choice benchmark dataset of difficult, but realistic questions that a caseworker might receive. Next, using these benchmark questions and corresponding expert-verified answers, we conducted a randomized experiment with caseworkers recruited from nonprofit outreach organizations in Los Angeles. Caseworkers in the control condition did not see chatbot suggestions and had a mean accuracy of 49%. Caseworkers in the treatment condition saw chatbot suggestions that we artificially varied to range in aggregate accuracy from low (53%) to high (100%). Caseworker performance significantly improves as chatbot quality improves: high-quality chatbots (96-100% accurate) improved caseworker accuracy by 27 percentage points. At the question-level, incorrect chatbot suggestions substantially reduce caseworker accuracy, with a two-thirds reduction on easy questions where the control group performed best (without chatbot suggestions). Finally, improvements in caseworker accuracy level off as chatbot accuracy increases, a phenomenon that we call the "AI underreliance plateau," which is a concern for real-world deployment and highlights the importance of evaluating human-in-the-loop tools with their users.

Related papers

• A Model-based Testing Technique for Amazon Lex Task-based Chatbots [46.12795846459242]
  LexTester is an automated model-based testing technique for Amazon Lex chatbots.<n>It generates a Dialog Graph of all possible interactions, from which an executable test suite is generated.
  arXiv  Detail & Related papers  (2026-07-01T15:48:43Z)
• Evaluating Commercial AI Chatbots as News Intermediaries [85.32040752972836]
  Best systems achieve over 90% multiple-choice accuracy on questions about events reported hours earlier.<n>Every model achieves its lowest accuracy on Hindi.<n> retrieval, not reasoning, failures drive over 70% of all errors.
  arXiv  Detail & Related papers  (2026-05-21T17:42:07Z)
• Automated Testing of Task-based Chatbots: How Far Are We? [5.64612424709862]
  Task-based chatbots are software, typically embedded in real-world applications, that assist users in completing tasks through a conversational interface.<n>In this paper, we evaluate the effectiveness of state-of-the-art testing techniques on a curated selection of task-based chatbots from GitHub.
  arXiv  Detail & Related papers  (2026-02-13T16:32:50Z)
• Development and Evaluation of HopeBot: an LLM-based chatbot for structured and interactive PHQ-9 depression screening [48.355615275247786]
  HopeBot administers the Patient Health Questionnaire-9 (PHQ-9) using retrieval-augmented generation and real-time clarification.<n>In a within-subject study, 132 adults in the United Kingdom and China completed both self-administered and chatbots versions.<n>Overall, 87.1% expressed willingness to reuse or recommend HopeBot.
  arXiv  Detail & Related papers  (2025-07-08T13:41:22Z)
• Empathetic Response in Audio-Visual Conversations Using Emotion Preference Optimization and MambaCompressor [44.499778745131046]
  Our study introduces a dual approach: firstly, we employ Emotional Preference Optimization (EPO) to train chatbots.<n>This training enables the model to discern fine distinctions between correct and counter-emotional responses.<n> Secondly, we introduce MambaCompressor to effectively compress and manage extensive conversation histories.<n>Our comprehensive experiments across multiple datasets demonstrate that our model significantly outperforms existing models in generating empathetic responses and managing lengthy dialogues.
  arXiv  Detail & Related papers  (2024-12-23T13:44:51Z)
• Prompt Engineering a Schizophrenia Chatbot: Utilizing a Multi-Agent Approach for Enhanced Compliance with Prompt Instructions [0.0699049312989311]
  Patients with schizophrenia often present with cognitive impairments that may hinder their ability to learn about their condition. While Large Language Models (LLMs) have the potential to make topical mental health information more accessible and engaging, their black-box nature raises concerns about ethics and safety.
  arXiv  Detail & Related papers  (2024-10-10T09:49:24Z)
• In Generative AI we Trust: Can Chatbots Effectively Verify Political Information? [39.58317527488534]
  This article presents a comparative analysis of the ability of two large language model (LLM)-based chatbots, ChatGPT and Bing Chat, to detect veracity of political information. We use AI auditing methodology to investigate how chatbots evaluate true, false, and borderline statements on five topics: COVID-19, Russian aggression against Ukraine, the Holocaust, climate change, and LGBTQ+ related debates. The results show high performance of ChatGPT for the baseline veracity evaluation task, with 72 percent of the cases evaluated correctly on average across languages without pre-training.
  arXiv  Detail & Related papers  (2023-12-20T15:17:03Z)
• Adding guardrails to advanced chatbots [5.203329540700177]
  Launch of ChatGPT in November 2022 has ushered in a new era of AI. There are already concerns that humans may be replaced by chatbots for a variety of jobs. These biases may cause significant harm and/or inequity toward different subpopulations.
  arXiv  Detail & Related papers  (2023-06-13T02:23:04Z)
• Is ChatGPT a General-Purpose Natural Language Processing Task Solver? [113.22611481694825]
  Large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot. Recently, the debut of ChatGPT has drawn a great deal of attention from the natural language processing (NLP) community. It is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot.
  arXiv  Detail & Related papers  (2023-02-08T09:44:51Z)
• A Deep Learning Approach to Integrate Human-Level Understanding in a Chatbot [0.4632366780742501]
  Unlike humans, chatbots can serve multiple customers at a time, are available 24/7 and reply in less than a fraction of a second. We performed sentiment analysis, emotion detection, intent classification and named-entity recognition using deep learning to develop chatbots with humanistic understanding and intelligence.
  arXiv  Detail & Related papers  (2021-12-31T22:26:41Z)
• Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention [55.77218465471519]
  This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework included a guiding robot and an interlocutor model that plays the role of humans. We examined our framework using three experimental setups and evaluate the guiding robot with four different metrics to demonstrated flexibility and performance advantages.
  arXiv  Detail & Related papers  (2021-03-30T15:24:37Z)
• Towards a Human-like Open-Domain Chatbot [56.94906081225566]
  This 2.6B parameter neural network is simply trained to minimize perplexity of the next token. We propose a human evaluation metric called Sensibleness and Specificity Average (SSA) which captures key elements of a human-like multi-turn conversation.
  arXiv  Detail & Related papers  (2020-01-27T18:53:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.