Navigating User Experience of ChatGPT-based Conversational Recommender Systems: The Effects of Prompt Guidance and Recommendation Domain

• URL: http://arxiv.org/abs/2405.13560v1
• Date: Wed, 22 May 2024 11:49:40 GMT
• Title: Navigating User Experience of ChatGPT-based Conversational Recommender Systems: The Effects of Prompt Guidance and Recommendation Domain
• Authors: Yizhe Zhang, Yucheng Jin, Li Chen, Ting Yang,
• Abstract summary: This study investigates the impact of prompt guidance (PG) and recommendation domain (RD) on the overall user experience of the system. The findings reveal that PG can substantially enhance the system's explainability, adaptability, perceived ease of use, and transparency.
• Score: 15.179413273734761
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conversational recommender systems (CRS) enable users to articulate their preferences and provide feedback through natural language. With the advent of large language models (LLMs), the potential to enhance user engagement with CRS and augment the recommendation process with LLM-generated content has received increasing attention. However, the efficacy of LLM-powered CRS is contingent upon the use of prompts, and the subjective perception of recommendation quality can differ across various recommendation domains. Therefore, we have developed a ChatGPT-based CRS to investigate the impact of these two factors, prompt guidance (PG) and recommendation domain (RD), on the overall user experience of the system. We conducted an online empirical study (N = 100) by employing a mixed-method approach that utilized a between-subjects design for the variable of PG (with vs. without) and a within-subjects design for RD (book recommendations vs. job recommendations). The findings reveal that PG can substantially enhance the system's explainability, adaptability, perceived ease of use, and transparency. Moreover, users are inclined to perceive a greater sense of novelty and demonstrate a higher propensity to engage with and try recommended items in the context of book recommendations as opposed to job recommendations. Furthermore, the influence of PG on certain user experience metrics and interactive behaviors appears to be modulated by the recommendation domain, as evidenced by the interaction effects between the two examined factors. This work contributes to the user-centered evaluation of ChatGPT-based CRS by investigating two prominent factors and offers practical design guidance.

Related papers

• LLM-Powered Explanations: Unraveling Recommendations Through Subgraph Reasoning [40.53821858897774]
  We introduce a novel recommender that synergies Large Language Models (LLMs) and Knowledge Graphs (KGs) to enhance the recommendation and provide interpretable results. Our approach significantly enhances both the effectiveness and interpretability of recommender systems.
  arXiv  Detail & Related papers  (2024-06-22T14:14:03Z)
• RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM Agents [30.250555783628762]
  This research argues that addressing these issues is not solely the recommender systems' responsibility. We introduce the RAH Recommender system, Assistant, and Human framework, emphasizing the alignment with user personalities. Our contributions provide a human-centered recommendation framework that partners effectively with various recommendation models.
  arXiv  Detail & Related papers  (2023-08-19T04:46:01Z)
• Rethinking the Evaluation for Conversational Recommendation in the Era of Large Language Models [115.7508325840751]
  The recent success of large language models (LLMs) has shown great potential to develop more powerful conversational recommender systems (CRSs) In this paper, we embark on an investigation into the utilization of ChatGPT for conversational recommendation, revealing the inadequacy of the existing evaluation protocol. We propose an interactive Evaluation approach based on LLMs named iEvaLM that harnesses LLM-based user simulators.
  arXiv  Detail & Related papers  (2023-05-22T15:12:43Z)
• Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System [11.404192885921498]
  Chat-Rec is a new paradigm for building conversational recommender systems. Chat-Rec is effective in learning user preferences and establishing connections between users and products. In experiments, Chat-Rec effectively improve the results of top-k recommendations and performs better in zero-shot rating prediction task.
  arXiv  Detail & Related papers  (2023-03-25T17:37:43Z)
• Aligning Recommendation and Conversation via Dual Imitation [56.236932446280825]
  We propose DICR (Dual Imitation for Conversational Recommendation), which designs a dual imitation to explicitly align the recommendation paths and user interest shift paths. By exchanging alignment signals, DICR achieves bidirectional promotion between recommendation and conversation modules. Experiments demonstrate that DICR outperforms the state-of-the-art models on recommendation and conversation performance with automatic, human, and novel explainability metrics.
  arXiv  Detail & Related papers  (2022-11-05T08:13:46Z)
• KECRS: Towards Knowledge-Enriched Conversational Recommendation System [50.0292306485452]
  chit-chat-based conversational recommendation systems (CRS) provide item recommendations to users through natural language interactions. external knowledge graphs (KG) have been introduced into chit-chat-based CRS. We propose the Knowledge-Enriched Conversational Recommendation System (KECRS) Experimental results on a large-scale dataset demonstrate that KECRS outperforms state-of-the-art chit-chat-based CRS.
  arXiv  Detail & Related papers  (2021-05-18T03:52:06Z)
• Learning to Ask Appropriate Questions in Conversational Recommendation [49.31942688227828]
  We propose the Knowledge-Based Question Generation System (KBQG), a novel framework for conversational recommendation. KBQG models a user's preference in a finer granularity by identifying the most relevant relations from a structured knowledge graph. Finially, accurate recommendations can be generated in fewer conversational turns.
  arXiv  Detail & Related papers  (2021-05-11T03:58:10Z)
• Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion [77.21442487537139]
  Conversational recommender systems (CRS) aim to recommend high-quality items to users through interactive conversations. First, the conversation data itself lacks of sufficient contextual information for accurately understanding users' preference. Second, there is a semantic gap between natural language expression and item-level user preference.
  arXiv  Detail & Related papers  (2020-07-08T11:14:23Z)
• Reward Constrained Interactive Recommendation with Natural Language Feedback [158.8095688415973]
  We propose a novel constraint-augmented reinforcement learning (RL) framework to efficiently incorporate user preferences over time. Specifically, we leverage a discriminator to detect recommendations violating user historical preference. Our proposed framework is general and is further extended to the task of constrained text generation.
  arXiv  Detail & Related papers  (2020-05-04T16:23:34Z)
• A Survey on Conversational Recommender Systems [11.319431345375751]
  Conversational recommender systems (CRS) take a different approach and support a richer set of interactions. The interest in CRS has significantly increased in the past few years. This development is mainly due to the significant progress in the area of natural language processing.
  arXiv  Detail & Related papers  (2020-04-01T18:00:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.