Large Language Models as Zero-Shot Conversational Recommenders

• URL: http://arxiv.org/abs/2308.10053v1
• Date: Sat, 19 Aug 2023 15:29:45 GMT
• Title: Large Language Models as Zero-Shot Conversational Recommenders
• Authors: Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, Julian McAuley
• Abstract summary: We present empirical studies on conversational recommendation tasks using representative large language models in a zero-shot setting. We construct a new dataset of recommendation-related conversations by scraping a popular discussion website. We observe that even without fine-tuning, large language models can outperform existing fine-tuned conversational recommendation models.
• Score: 52.57230221644014
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we present empirical studies on conversational recommendation tasks using representative large language models in a zero-shot setting with three primary contributions. (1) Data: To gain insights into model behavior in "in-the-wild" conversational recommendation scenarios, we construct a new dataset of recommendation-related conversations by scraping a popular discussion website. This is the largest public real-world conversational recommendation dataset to date. (2) Evaluation: On the new dataset and two existing conversational recommendation datasets, we observe that even without fine-tuning, large language models can outperform existing fine-tuned conversational recommendation models. (3) Analysis: We propose various probing tasks to investigate the mechanisms behind the remarkable performance of large language models in conversational recommendation. We analyze both the large language models' behaviors and the characteristics of the datasets, providing a holistic understanding of the models' effectiveness, limitations and suggesting directions for the design of future conversational recommenders

Related papers

• Improving Context-Aware Preference Modeling for Language Models [62.32080105403915]
  We consider the two-step preference modeling procedure that first resolves the under-specification by selecting a context, and then evaluates preference with respect to the chosen context. We contribute context-conditioned preference datasets and experiments that investigate the ability of language models to evaluate context-specific preference.
  arXiv  Detail & Related papers  (2024-07-20T16:05:17Z)
• Evaluating Large Language Models as Generative User Simulators for Conversational Recommendation [20.171574438536673]
  We introduce a new protocol to measure the degree to which language models can accurately emulate human behavior in conversational recommendation. We demonstrate these tasks effectively reveal deviations of language models from human behavior, and offer insights on how to reduce the deviations with model selection and prompting strategies.
  arXiv  Detail & Related papers  (2024-03-13T18:16:21Z)
• A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation [5.661396828160973]
  We conduct an empirical analysis of conversational large language models in generating natural language text from semantic triples. We compare four large language models of varying sizes with different prompting techniques. Our findings show that the capabilities of large language models in triple verbalization can be significantly improved through few-shot prompting, post-processing, and efficient fine-tuning techniques.
  arXiv  Detail & Related papers  (2024-02-02T15:26:39Z)
• Instructive Dialogue Summarization with Query Aggregations [41.89962538701501]
  We introduce instruction-finetuned language models to expand the capability set of dialogue summarization models. We propose a three-step approach to synthesize high-quality query-based summarization triples. By training a unified model called InstructDS on three summarization datasets with multi-purpose instructive triples, we expand the capability of dialogue summarization models.
  arXiv  Detail & Related papers  (2023-10-17T04:03:00Z)
• AUGUST: an Automatic Generation Understudy for Synthesizing Conversational Recommendation Datasets [56.052803235932686]
  We propose a novel automatic dataset synthesis approach that can generate both large-scale and high-quality recommendation dialogues. In doing so, we exploit: (i) rich personalized user profiles from traditional recommendation datasets, (ii) rich external knowledge from knowledge graphs, and (iii) the conversation ability contained in human-to-human conversational recommendation datasets.
  arXiv  Detail & Related papers  (2023-06-16T05:27:14Z)
• Talk the Walk: Synthetic Data Generation for Conversational Music Recommendation [62.019437228000776]
  We present TalkWalk, which generates realistic high-quality conversational data by leveraging encoded expertise in widely available item collections. We generate over one million diverse conversations in a human-collected dataset.
  arXiv  Detail & Related papers  (2023-01-27T01:54:16Z)
• Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations [35.74225306947918]
  We propose a joint latent space learning and clustering framework built upon PLM embeddings. Our model effectively leverages the strong representation power and superb linguistic features brought by PLMs for topic discovery.
  arXiv  Detail & Related papers  (2022-02-09T17:26:08Z)
• ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining [61.82562838486632]
  We crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads. We benchmark state-of-the-art models on our datasets and analyze characteristics associated with the data.
  arXiv  Detail & Related papers  (2021-06-01T22:17:13Z)
• A Survey on Neural Recommendation: From Collaborative Filtering to Content and Context Enriched Recommendation [70.69134448863483]
  Research in recommendation has shifted to inventing new recommender models based on neural networks. In recent years, we have witnessed significant progress in developing neural recommender models.
  arXiv  Detail & Related papers  (2021-04-27T08:03:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.