WaLLM -- Insights from an LLM-Powered Chatbot deployment via WhatsApp

• URL: http://arxiv.org/abs/2505.08894v1
• Date: Tue, 13 May 2025 18:36:18 GMT
• Title: WaLLM -- Insights from an LLM-Powered Chatbot deployment via WhatsApp
• Abstract summary: In developing regions, access to information remains a challenge due to persistent digital divide.<n>We developed WaLLM - a custom AI bot over WhatsApp.<n>Our service has been operational for over 6 months, amassing over 14.7K queries from approximately 100 users.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in generative AI, such as ChatGPT, have transformed access to information in education, knowledge-seeking, and everyday decision-making. However, in many developing regions, access remains a challenge due to the persistent digital divide. To help bridge this gap, we developed WaLLM - a custom AI chatbot over WhatsApp, a widely used communication platform in developing regions. Beyond answering queries, WaLLM offers several features to enhance user engagement: a daily top question, suggested follow-up questions, trending and recent queries, and a leaderboard-based reward system. Our service has been operational for over 6 months, amassing over 14.7K queries from approximately 100 users. In this paper, we present WaLLM's design and a systematic analysis of logs to understand user interactions. Our results show that 55% of user queries seek factual information. "Health and well-being" was the most popular topic (28%), including queries about nutrition and disease, suggesting users view WaLLM as a reliable source. Two-thirds of users' activity occurred within 24 hours of the daily top question. Users who accessed the "Leaderboard" interacted with WaLLM 3x as those who did not. We conclude by discussing implications for culture-based customization, user interface design, and appropriate calibration of users' trust in AI systems for developing regions.

Related papers

• Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Learning Context using Brain Sensing [66.03354630509942]
  This paper reports on a study where we compare three designs for user-AI interaction in a learning context.<n>Fifty study participants were tasked with learning about nuclear safety protocols.
  arXiv  Detail & Related papers  (2026-09-01T02:24:35Z)
• Three Years of r/ChatGPT: Societal Impact Evaluations from Social Media Data [22.976622021605333]
  ChatGPT was launched on November 30, 2022; the r/ChatGPT subreddit was created just one day later.<n>We use social media as a data source for understanding the societal impact of widely-adopted consumer AI products.<n>We show that PuLSE can detect the increase in emotional engagement as early as October 2024.
  arXiv  Detail & Related papers  (2026-06-04T06:23:41Z)
• SimpleDevQA: Benchmarking Large Language Models on Development Knowledge QA [58.75982433502236]
  The Dev Knowledge QA task accounts for 39.6% of interactions.<n>Only 27.5% of real Dev Knowledge QA dialogues focus on code understanding.<n>Only 17.1% of real-world Dev Knowledge QA dialogues can be used for constructing a benchmark.
  arXiv  Detail & Related papers  (2025-12-09T17:58:36Z)
• User Feedback in Human-LLM Dialogues: A Lens to Understand Users But Noisy as a Learning Signal [58.43749783815486]
  We study implicit user feedback in two user-LM interaction datasets.<n>We find that the contents of user feedback can improve model performance in short human-designed questions.<n>We also find that the usefulness of user feedback is largely tied to the quality of the user's initial prompt.
  arXiv  Detail & Related papers  (2025-07-30T23:33:29Z)
• Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scale [51.9706400130481]
  Large Language Models (LLMs) have emerged as personalized assistants for users across a wide range of tasks.<n> PERSONAMEM features curated user profiles with over 180 simulated user-LLM interaction histories.<n>We evaluate LLM chatbots' ability to identify the most suitable response according to the current state of the user's profile.
  arXiv  Detail & Related papers  (2025-04-19T08:16:10Z)
• Mind the Gap! Static and Interactive Evaluations of Large Audio Models [55.87220295533817]
  Large Audio Models (LAMs) are designed to power voice-native experiences.<n>This study introduces an interactive approach to evaluate LAMs and collect 7,500 LAM interactions from 484 participants.
  arXiv  Detail & Related papers  (2025-02-21T20:29:02Z)
• From Voice to Value: Leveraging AI to Enhance Spoken Online Reviews on the Go [21.811104609265158]
  We developed Vocalizer, a mobile application that enables users to provide reviews through voice input.<n>Our findings show that users frequently utilized the AI agent to add more detailed information to their reviews.<n>We also show how interactive AI features can improve users self-efficacy and willingness to share reviews online.
  arXiv  Detail & Related papers  (2024-12-06T21:59:47Z)
• Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations [8.848859080368799]
  Collaborative STORM lets users observe and steer the discourse among several LM agents. The agents ask questions on the user's behalf, allowing the user to discover unknown unknowns serendipitously. For automatic evaluation, we construct the WildSeek dataset by collecting real information-seeking records with user goals.
  arXiv  Detail & Related papers  (2024-08-27T17:50:03Z)
• Improving Ontology Requirements Engineering with OntoChat and Participatory Prompting [3.3241053483599563]
  ORE has primarily relied on manual methods, such as interviews and collaborative forums, to gather user requirements from domain experts. Current OntoChat offers a framework for ORE that utilise large language models (LLMs) to streamline the process. This study produces pre-defined prompt templates based on user queries, focusing on creating and refining personas, goals, scenarios, sample data, and data resources for user stories.
  arXiv  Detail & Related papers  (2024-08-09T19:21:14Z)
• WildChat: 1M ChatGPT Interaction Logs in the Wild [88.05964311416717]
  WildChat is a corpus of 1 million user-ChatGPT conversations, which consists of over 2.5 million interaction turns. In addition to timestamped chat transcripts, we enrich the dataset with demographic data, including state, country, and hashed IP addresses.
  arXiv  Detail & Related papers  (2024-05-02T17:00:02Z)
• Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference [48.99117537559644]
  We introduce Arena, an open platform for evaluating Large Language Models (LLMs) based on human preferences. Our methodology employs a pairwise comparison approach and leverages input from a diverse user base through crowdsourcing. This paper describes the platform, analyzes the data we have collected so far, and explains the tried-and-true statistical methods we are using.
  arXiv  Detail & Related papers  (2024-03-07T01:22:38Z)
• Unveiling Inclusiveness-Related User Feedback in Mobile Applications [7.212232917917022]
  We leverage user feedback from Reddit, Google Play Store, and X for 50 of the most popular apps in the world.<n>Using a Socio-Technical Grounded Theory approach, we analyzed 22,000 posts across the three sources.<n>We organize our results in a taxonomy for inclusiveness comprising 5 major categories: Algorithmic Bias, Technology, Demography, Accessibility, and Other Human Values.
  arXiv  Detail & Related papers  (2023-11-02T04:05:46Z)
• Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning [69.42679922160684]
  We propose feedback-weighted learning based on importance sampling to improve upon an initial supervised system using binary user feedback. Our work opens the prospect to exploit interactions with real users and improve conversational systems after deployment.
  arXiv  Detail & Related papers  (2020-11-01T19:50:34Z)
• Study of the usability of LinkedIn: a social media platform meant to connect employers and employees [91.3755431537592]
  This paper is assessing LinkedIn's usability using both user and expert evaluation. The overall usability of LinkedIn application has been measured by using SUS (System Usability Scale)
  arXiv  Detail & Related papers  (2020-06-06T18:19:45Z)
• Information Seeking in the Spirit of Learning: a Dataset for Conversational Curiosity [10.409312809724458]
  We design a Wizard-of-Oz dialog task that tests the hypothesis that engagement increases when users are presented with facts related to what they know. We collect and release 14K dialogs (181K utterances) where users and assistants converse about geographic topics. This dataset is annotated with pre-existing user knowledge, message-level dialog acts, grounding to Wikipedia, and user reactions to messages.
  arXiv  Detail & Related papers  (2020-05-01T01:55:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.