Reading Users' Minds from What They Say: An Investigation into LLM-based Empathic Mental Inference

• URL: http://arxiv.org/abs/2403.13301v1
• Date: Wed, 20 Mar 2024 04:57:32 GMT
• Title: Reading Users' Minds from What They Say: An Investigation into LLM-based Empathic Mental Inference
• Authors: Qihao Zhu, Leah Chong, Maria Yang, Jianxi Luo,
• Abstract summary: In human-centered design, developing a comprehensive and in-depth understanding of user experiences is paramount. accurately comprehending the real underlying mental states of a large human population remains a significant challenge today. This paper investigates the use of Large Language Models (LLMs) for performing mental inference tasks.
• Score: 6.208698652041961
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In human-centered design, developing a comprehensive and in-depth understanding of user experiences, i.e., empathic understanding, is paramount for designing products that truly meet human needs. Nevertheless, accurately comprehending the real underlying mental states of a large human population remains a significant challenge today. This difficulty mainly arises from the trade-off between depth and scale of user experience research: gaining in-depth insights from a small group of users does not easily scale to a larger population, and vice versa. This paper investigates the use of Large Language Models (LLMs) for performing mental inference tasks, specifically inferring users' underlying goals and fundamental psychological needs (FPNs). Baseline and benchmark datasets were collected from human users and designers to develop an empathic accuracy metric for measuring the mental inference performance of LLMs. The empathic accuracy of inferring goals and FPNs of different LLMs with varied zero-shot prompt engineering techniques are experimented against that of human designers. Experimental results suggest that LLMs can infer and understand the underlying goals and FPNs of users with performance comparable to that of human designers, suggesting a promising avenue for enhancing the scalability of empathic design approaches through the integration of advanced artificial intelligence technologies. This work has the potential to significantly augment the toolkit available to designers during human-centered design, enabling the development of both large-scale and in-depth understanding of users' experiences.

Related papers

• PEFT-U: Parameter-Efficient Fine-Tuning for User Personalization [9.594958534074074]
  We introduce the PEFT-U Benchmark: a new dataset for building and evaluating NLP models for user personalization. We explore the challenge of efficiently personalizing LLMs to accommodate user-specific preferences in the context of diverse user-centered tasks.
  arXiv  Detail & Related papers  (2024-07-25T14:36:18Z)
• Predicting and Understanding Human Action Decisions: Insights from Large Language Models and Cognitive Instance-Based Learning [0.0]
  Large Language Models (LLMs) have demonstrated their capabilities across various tasks. This paper exploits the reasoning and generative capabilities of the LLMs to predict human behavior in two sequential decision-making tasks. We compare the performance of LLMs with a cognitive instance-based learning model, which imitates human experiential decision-making.
  arXiv  Detail & Related papers  (2024-07-12T14:13:06Z)
• CogErgLLM: Exploring Large Language Model Systems Design Perspective Using Cognitive Ergonomics [0.0]
  Integrating cognitive ergonomics with LLMs is essential for enhancing safety, reliability, and user satisfaction in human-AI interactions. Current LLM design often lacks this integration, leading to systems that may not fully align with human cognitive capabilities and limitations. Insufficient focus on incorporating cognitive science methods exacerbates biases in LLM outputs, while inconsistent application of user-centered design principles results in sub-optimal user experiences.
  arXiv  Detail & Related papers  (2024-07-03T07:59:52Z)
• Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinement [82.56964750522161]
  We introduce Persona-DB, a simple framework consisting of a hierarchical construction process to improve generalization across task contexts. In the task of response forecasting, Persona-DB demonstrates superior efficiency in maintaining accuracy with a significantly reduced retrieval size. Our experiments also indicate a marked improvement of over 15% under cold-start scenarios, when users have extremely sparse data.
  arXiv  Detail & Related papers  (2024-02-16T20:20:43Z)
• Interpreting Pretrained Language Models via Concept Bottlenecks [55.47515772358389]
  Pretrained language models (PLMs) have made significant strides in various natural language processing tasks. The lack of interpretability due to their black-box'' nature poses challenges for responsible implementation. We propose a novel approach to interpreting PLMs by employing high-level, meaningful concepts that are easily understandable for humans.
  arXiv  Detail & Related papers  (2023-11-08T20:41:18Z)
• The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions [114.67699010359637]
  We analyze a large-scale collection of real user queries to GPT. We find that tasks such as design'' and planning'' are prevalent in user interactions but are largely neglected or different from traditional NLP benchmarks.
  arXiv  Detail & Related papers  (2023-10-19T02:12:17Z)
• Aligning Large Language Models with Human: A Survey [53.6014921995006]
  Large Language Models (LLMs) trained on extensive textual corpora have emerged as leading solutions for a broad array of Natural Language Processing (NLP) tasks. Despite their notable performance, these models are prone to certain limitations such as misunderstanding human instructions, generating potentially biased content, or factually incorrect information. This survey presents a comprehensive overview of these alignment technologies, including the following aspects.
  arXiv  Detail & Related papers  (2023-07-24T17:44:58Z)
• Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration [116.09561564489799]
  Solo Performance Prompting transforms a single LLM into a cognitive synergist by engaging in multi-turn self-collaboration with multiple personas. A cognitive synergist is an intelligent agent that collaboratively combines multiple minds' strengths and knowledge to enhance problem-solving in complex tasks. Our in-depth analysis shows that assigning multiple fine-grained personas in LLMs improves problem-solving abilities compared to using a single or fixed number of personas.
  arXiv  Detail & Related papers  (2023-07-11T14:45:19Z)
• User profile-driven large-scale multi-agent learning from demonstration in federated human-robot collaborative environments [5.218882272051637]
  This paper introduces a novel user profile formulation for providing a fine-grained representation of the exhibited human behavior. The overall designed scheme enables both short- and long-term analysis/interpretation of the human behavior.
  arXiv  Detail & Related papers  (2021-03-30T15:33:21Z)
• Understanding the Usability Challenges of Machine Learning In High-Stakes Decision Making [67.72855777115772]
  Machine learning (ML) is being applied to a diverse and ever-growing set of domains. In many cases, domain experts -- who often have no expertise in ML or data science -- are asked to use ML predictions to make high-stakes decisions. We investigate the ML usability challenges present in the domain of child welfare screening through a series of collaborations with child welfare screeners.
  arXiv  Detail & Related papers  (2021-03-02T22:50:45Z)
• The Role of Individual User Differences in Interpretable and Explainable Machine Learning Systems [0.3169089186688223]
  We study how individual skills and personality traits predict interpretability, explainability, and knowledge discovery from machine learning generated model output. Our work relies on Fuzzy Trace Theory, a leading theory of how humans process numerical stimuli.
  arXiv  Detail & Related papers  (2020-09-14T18:15:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.