User Modeling Challenges in Interactive AI Assistant Systems

• URL: http://arxiv.org/abs/2403.20134v1
• Date: Fri, 29 Mar 2024 11:54:13 GMT
• Title: User Modeling Challenges in Interactive AI Assistant Systems
• Authors: Megan Su, Yuwei Bao,
• Abstract summary: Interactive Artificial Intelligent(AI) assistant systems are designed to offer timely guidance to help human users to complete a variety tasks. One of the remaining challenges is to understand user's mental states during the task for more personalized guidance. In this work, we analyze users' mental states during task executions and investigate the capabilities and challenges for large language models to interpret user profiles for more personalized user guidance.
• Score: 3.1204913702660475
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Interactive Artificial Intelligent(AI) assistant systems are designed to offer timely guidance to help human users to complete a variety tasks. One of the remaining challenges is to understand user's mental states during the task for more personalized guidance. In this work, we analyze users' mental states during task executions and investigate the capabilities and challenges for large language models to interpret user profiles for more personalized user guidance.

Related papers

• AIAP: A No-Code Workflow Builder for Non-Experts with Natural Language and Multi-Agent Collaboration [12.74618436015574]
  We introduce AIAP, a no-code platform that integrates natural language input with visual system complexity.<n>A user study involving 32 participants showed that AIAP's AI-generated suggestions, modular, and automatic identification of data, actions, and context significantly improved participants' ability to develop services intuitively.
  arXiv  Detail & Related papers  (2025-08-04T14:36:31Z)
• PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants [31.486658078902025]
  We introduce PersonaLens, a benchmark for evaluating personalization in task-oriented AI assistants.<n>Our benchmark features diverse user profiles equipped with rich preferences and interaction histories, along with two specialized LLM-based agents.<n>We reveal significant variability in their personalization capabilities, providing crucial insights for advancing conversational AI systems.
  arXiv  Detail & Related papers  (2025-06-11T16:16:07Z)
• RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Users [8.044364097415007]
  RealWebAssist is a novel benchmark designed to evaluate sequential instruction-following in realistic scenarios involving long-horizon interactions with the web. Each user instructs a web-based assistant to perform a series of tasks on multiple websites. A successful agent must reason about the true intent behind each instruction, keep track of the mental state of the user, understand user-specific routines, and ground the intended tasks to actions on the correct GUI elements.
  arXiv  Detail & Related papers  (2025-04-14T17:36:46Z)
• Proactive User Information Acquisition via Chats on User-Favored Topics [3.6698472838681893]
  This study proposes the PIVOT task, designed to advance the technical foundation for these systems. We found that even recent large language models (LLMs) show a low success rate in the PIVOT task. We developed a simple but effective system for this task by incorporating insights obtained through the analysis of this dataset.
  arXiv  Detail & Related papers  (2025-04-10T12:32:16Z)
• Enhancing Discoverability in Enterprise Conversational Systems with Proactive Question Suggestions [5.356008176627551]
  This paper proposes a framework to enhance question suggestions in conversational enterprise AI systems. Our approach combines periodic user intent analysis at the population level with chat session-based question generation. We evaluate the framework using real-world data from the AI Assistant for Adobe Experience Platform.
  arXiv  Detail & Related papers  (2024-12-14T19:04:16Z)
• Survey of User Interface Design and Interaction Techniques in Generative AI Applications [79.55963742878684]
  We aim to create a compendium of different user-interaction patterns that can be used as a reference for designers and developers alike. We also strive to lower the entry barrier for those attempting to learn more about the design of generative AI applications.
  arXiv  Detail & Related papers  (2024-10-28T23:10:06Z)
• Data Analysis in the Era of Generative AI [56.44807642944589]
  This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.
  arXiv  Detail & Related papers  (2024-09-27T06:31:03Z)
• Supporting Experts with a Multimodal Machine-Learning-Based Tool for Human Behavior Analysis of Conversational Videos [40.30407535831779]
  We developed Providence, a visual-programming-based tool based on design considerations derived from a formative study with experts. It enables experts to combine various machine learning algorithms to capture human behavioral cues without writing code. Our study showed its preferable usability and satisfactory output with less cognitive load imposed in accomplishing scene search tasks of conversations.
  arXiv  Detail & Related papers  (2024-02-17T00:27:04Z)
• Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents [110.25679611755962]
  Current language model-driven agents often lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions. We introduce Intention-in-Interaction (IN3), a novel benchmark designed to inspect users' implicit intentions through explicit queries. We empirically train Mistral-Interact, a powerful model that proactively assesses task vagueness, inquires user intentions, and refines them into actionable goals.
  arXiv  Detail & Related papers  (2024-02-14T14:36:30Z)
• Can Foundation Models Watch, Talk and Guide You Step by Step to Make a Cake? [62.59699229202307]
  Despite advances in AI, it remains a significant challenge to develop interactive task guidance systems. We created a new multimodal benchmark dataset, Watch, Talk and Guide (WTaG) based on natural interaction between a human user and a human instructor. We leveraged several foundation models to study to what extent these models can be quickly adapted to perceptually enabled task guidance.
  arXiv  Detail & Related papers  (2023-11-01T15:13:49Z)
• When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities [60.5609416496429]
  The capability of large language models has been dramatically improved. Such a major leap-forward in general AI capacity will change the pattern of how personalization is conducted. By leveraging large language models as general-purpose interface, personalization systems may compile user requests into plans.
  arXiv  Detail & Related papers  (2023-07-31T02:48:56Z)
• User Simulation for Evaluating Information Access Systems [38.48048183731099]
  evaluating the effectiveness of interactive intelligent systems is a complex scientific challenge. This book provides a thorough understanding of user simulation techniques designed specifically for evaluation. It covers both general frameworks for designing user simulators, and specific models and algorithms for simulating user interactions with search engines, recommender systems, and conversational assistants.
  arXiv  Detail & Related papers  (2023-06-14T14:54:06Z)
• Human in the loop approaches in multi-modal conversational task guidance system development [6.493148232868973]
  Development of task guidance systems for aiding humans in a situated task remains a challenging problem. We first highlight some of the challenges involved during the development of such systems. We then provide an overview of existing datasets available and highlight their limitations.
  arXiv  Detail & Related papers  (2022-11-03T14:05:30Z)
• What Do End-Users Really Want? Investigation of Human-Centered XAI for Mobile Health Apps [69.53730499849023]
  We present a user-centered persona concept to evaluate explainable AI (XAI) Results show that users' demographics and personality, as well as the type of explanation, impact explanation preferences. Our insights bring an interactive, human-centered XAI closer to practical application.
  arXiv  Detail & Related papers  (2022-10-07T12:51:27Z)
• Are Akpans Trick or Treat: Unveiling Helpful Biases in Assistant Systems [55.09907990139756]
  Information-seeking AI assistant systems aim to answer users' queries about knowledge in a timely manner. In this paper, we study computational measurements of helpfulness. Experiments with state-of-the-art dialogue systems reveal that existing systems tend to be more helpful for questions regarding concepts from highly-developed countries.
  arXiv  Detail & Related papers  (2022-05-25T07:58:38Z)
• Multipurpose Intelligent Process Automation via Conversational Assistant [3.808063547958558]
  Intelligent Process Automation (IPA) is an emerging technology with a primary goal to assist the knowledge worker by taking care of repetitive, routine and low-cognitive tasks. We tackle a challenge of implementing an IPA conversational assistant in a real-world industrial setting with a lack of structured training data. Our proposed system brings two significant benefits: First, it reduces repetitive and time-consuming activities and, therefore, allows workers to focus on more intelligent processes.
  arXiv  Detail & Related papers  (2020-01-07T21:47:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.