Navigating User Experience of ChatGPT-based Conversational Recommender Systems: The Effects of Prompt Guidance and Recommendation Domain

• URL: http://arxiv.org/abs/2405.13560v1
• Date: Wed, 22 May 2024 11:49:40 GMT
• Title: Navigating User Experience of ChatGPT-based Conversational Recommender Systems: The Effects of Prompt Guidance and Recommendation Domain
• Authors: Yizhe Zhang, Yucheng Jin, Li Chen, Ting Yang,
• Abstract summary: This study investigates the impact of prompt guidance (PG) and recommendation domain (RD) on the overall user experience of the system. The findings reveal that PG can substantially enhance the system's explainability, adaptability, perceived ease of use, and transparency.
• Score: 15.179413273734761
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conversational recommender systems (CRS) enable users to articulate their preferences and provide feedback through natural language. With the advent of large language models (LLMs), the potential to enhance user engagement with CRS and augment the recommendation process with LLM-generated content has received increasing attention. However, the efficacy of LLM-powered CRS is contingent upon the use of prompts, and the subjective perception of recommendation quality can differ across various recommendation domains. Therefore, we have developed a ChatGPT-based CRS to investigate the impact of these two factors, prompt guidance (PG) and recommendation domain (RD), on the overall user experience of the system. We conducted an online empirical study (N = 100) by employing a mixed-method approach that utilized a between-subjects design for the variable of PG (with vs. without) and a within-subjects design for RD (book recommendations vs. job recommendations). The findings reveal that PG can substantially enhance the system's explainability, adaptability, perceived ease of use, and transparency. Moreover, users are inclined to perceive a greater sense of novelty and demonstrate a higher propensity to engage with and try recommended items in the context of book recommendations as opposed to job recommendations. Furthermore, the influence of PG on certain user experience metrics and interactive behaviors appears to be modulated by the recommendation domain, as evidenced by the interaction effects between the two examined factors. This work contributes to the user-centered evaluation of ChatGPT-based CRS by investigating two prominent factors and offers practical design guidance.

Related papers

• Search-Based Interaction For Conversation Recommendation via Generative Reward Model Based Simulated User [117.82681846559909]
  Conversational recommendation systems (CRSs) use multi-turn interaction to capture user preferences and provide personalized recommendations. We propose a generative reward model based simulated user, named GRSU, for automatic interaction with CRSs.
  arXiv  Detail & Related papers  (2025-04-29T06:37:30Z)
• Should We Tailor the Talk? Understanding the Impact of Conversational Styles on Preference Elicitation in Conversational Recommender Systems [19.830560938115436]
  This study investigates the impact of different conversational styles on preference elicitation, task performance, and user satisfaction with Conversational recommender systems (CRSs) Our results indicate that adapting conversational strategies based on user expertise and allowing flexibility between styles can enhance both user satisfaction and the effectiveness of recommendations in CRSs.
  arXiv  Detail & Related papers  (2025-04-17T17:01:17Z)
• Exploring the Impact of Personality Traits on Conversational Recommender Systems: A Simulation with Large Language Models [70.180385882195]
  This paper introduces a personality-aware user simulation for Conversational Recommender Systems (CRSs) The user agent induces customizable personality traits and preferences, while the system agent possesses the persuasion capability to simulate realistic interaction in CRSs. Experimental results demonstrate that state-of-the-art LLMs can effectively generate diverse user responses aligned with specified personality traits.
  arXiv  Detail & Related papers  (2025-04-09T13:21:17Z)
• Graph Retrieval-Augmented LLM for Conversational Recommendation Systems [52.35491420330534]
  G-CRS (Graph Retrieval-Augmented Large Language Model for Conversational Recommender Systems) is a training-free framework that combines graph retrieval-augmented generation and in-context learning. G-CRS achieves superior recommendation performance compared to existing methods without requiring task-specific training.
  arXiv  Detail & Related papers  (2025-03-09T03:56:22Z)
• LLM-Powered Explanations: Unraveling Recommendations Through Subgraph Reasoning [40.53821858897774]
  We introduce a novel recommender that synergies Large Language Models (LLMs) and Knowledge Graphs (KGs) to enhance the recommendation and provide interpretable results. Our approach significantly enhances both the effectiveness and interpretability of recommender systems.
  arXiv  Detail & Related papers  (2024-06-22T14:14:03Z)
• RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM Agents [30.250555783628762]
  This research argues that addressing these issues is not solely the recommender systems' responsibility. We introduce the RAH Recommender system, Assistant, and Human framework, emphasizing the alignment with user personalities. Our contributions provide a human-centered recommendation framework that partners effectively with various recommendation models.
  arXiv  Detail & Related papers  (2023-08-19T04:46:01Z)
• Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models [115.7508325840751]
  The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs) In this paper, we embark on an investigation into the utilization of ChatGPT for conversational recommendation, revealing the inadequacy of the existing evaluation protocol. We propose an interactive Evaluation approach based on LLMs named iEvaLM that harnesses LLM-based user simulators.
  arXiv  Detail & Related papers  (2023-05-22T15:12:43Z)
• Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System [11.404192885921498]
  Chat-Rec is a new paradigm for building conversational recommender systems. Chat-Rec is effective in learning user preferences and establishing connections between users and products. In experiments, Chat-Rec effectively improve the results of top-k recommendations and performs better in zero-shot rating prediction task.
  arXiv  Detail & Related papers  (2023-03-25T17:37:43Z)
• Aligning Recommendation and Conversation via Dual Imitation [56.236932446280825]
  We propose DICR (Dual Imitation for Conversational Recommendation), which designs a dual imitation to explicitly align the recommendation paths and user interest shift paths. By exchanging alignment signals, DICR achieves bidirectional promotion between recommendation and conversation modules. Experiments demonstrate that DICR outperforms the state-of-the-art models on recommendation and conversation performance with automatic, human, and novel explainability metrics.
  arXiv  Detail & Related papers  (2022-11-05T08:13:46Z)
• KECRS: Towards Knowledge-Enriched Conversational Recommendation System [50.0292306485452]
  chit-chat-based conversational recommendation systems (CRS) provide item recommendations to users through natural language interactions. external knowledge graphs (KG) have been introduced into chit-chat-based CRS. We propose the Knowledge-Enriched Conversational Recommendation System (KECRS) Experimental results on a large-scale dataset demonstrate that KECRS outperforms state-of-the-art chit-chat-based CRS.
  arXiv  Detail & Related papers  (2021-05-18T03:52:06Z)
• Learning to Ask Appropriate Questions in Conversational Recommendation [49.31942688227828]
  We propose the Knowledge-Based Question Generation System (KBQG), a novel framework for conversational recommendation. KBQG models a user's preference in a finer granularity by identifying the most relevant relations from a structured knowledge graph. Finially, accurate recommendations can be generated in fewer conversational turns.
  arXiv  Detail & Related papers  (2021-05-11T03:58:10Z)
• Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion [77.21442487537139]
  Conversational recommender systems (CRS) aim to recommend high-quality items to users through interactive conversations. First, the conversation data itself lacks of sufficient contextual information for accurately understanding users' preference. Second, there is a semantic gap between natural language expression and item-level user preference.
  arXiv  Detail & Related papers  (2020-07-08T11:14:23Z)
• Reward Constrained Interactive Recommendation with Natural Language Feedback [158.8095688415973]
  We propose a novel constraint-augmented reinforcement learning (RL) framework to efficiently incorporate user preferences over time. Specifically, we leverage a discriminator to detect recommendations violating user historical preference. Our proposed framework is general and is further extended to the task of constrained text generation.
  arXiv  Detail & Related papers  (2020-05-04T16:23:34Z)
• A Survey on Conversational Recommender Systems [11.319431345375751]
  Conversational recommender systems (CRS) take a different approach and support a richer set of interactions. The interest in CRS has significantly increased in the past few years. This development is mainly due to the significant progress in the area of natural language processing.
  arXiv  Detail & Related papers  (2020-04-01T18:00:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.