Integrating Summarization and Retrieval for Enhanced Personalization via
Large Language Models
- URL: http://arxiv.org/abs/2310.20081v1
- Date: Mon, 30 Oct 2023 23:40:41 GMT
- Title: Integrating Summarization and Retrieval for Enhanced Personalization via
Large Language Models
- Authors: Chris Richardson, Yao Zhang, Kellen Gillespie, Sudipta Kar, Arshdeep
Singh, Zeynab Raeesy, Omar Zia Khan, Abhinav Sethy
- Abstract summary: Personalization is an essential factor in user experience with natural language processing (NLP) systems.
With the emergence of Large Language Models (LLMs), a key question is how to leverage these models to better personalize user experiences.
We propose a novel summary-augmented personalization with task-aware user summaries generated by LLMs.
- Score: 11.950478880423733
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Personalization, the ability to tailor a system to individual users, is an
essential factor in user experience with natural language processing (NLP)
systems. With the emergence of Large Language Models (LLMs), a key question is
how to leverage these models to better personalize user experiences. To
personalize a language model's output, a straightforward approach is to
incorporate past user data into the language model prompt, but this approach
can result in lengthy inputs exceeding limitations on input length and
incurring latency and cost issues. Existing approaches tackle such challenges
by selectively extracting relevant user data (i.e. selective retrieval) to
construct a prompt for downstream tasks. However, retrieval-based methods are
limited by potential information loss, lack of more profound user
understanding, and cold-start challenges. To overcome these limitations, we
propose a novel summary-augmented approach by extending retrieval-augmented
personalization with task-aware user summaries generated by LLMs. The summaries
can be generated and stored offline, enabling real-world systems with runtime
constraints like voice assistants to leverage the power of LLMs. Experiments
show our method with 75% less of retrieved user data is on-par or outperforms
retrieval augmentation on most tasks in the LaMP personalization benchmark. We
demonstrate that offline summarization via LLMs and runtime retrieval enables
better performance for personalization on a range of tasks under practical
constraints.
Related papers
- SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning [70.21358720599821]
Large language models (LLMs) hold the promise of solving diverse tasks when provided with appropriate natural language prompts.
We propose SELF-GUIDE, a multi-stage mechanism in which we synthesize task-specific input-output pairs from the student LLM.
We report an absolute improvement of approximately 15% for classification tasks and 18% for generation tasks in the benchmark's metrics.
arXiv Detail & Related papers (2024-07-16T04:41:58Z) - Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models [66.24055500785657]
Traditional turn-based chat systems prevent users from verbally interacting with system while it is generating responses.
To overcome these limitations, we adapt existing LLMs to listen users while generating output and provide users with instant feedback.
We build a dataset consisting of alternating time slices of queries and responses as well as covering typical feedback types in instantaneous interactions.
arXiv Detail & Related papers (2024-06-22T03:20:10Z) - Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? [54.667202878390526]
Long-context language models (LCLMs) have the potential to revolutionize our approach to tasks traditionally reliant on external tools like retrieval systems or databases.
We introduce LOFT, a benchmark of real-world tasks requiring context up to millions of tokens designed to evaluate LCLMs' performance on in-context retrieval and reasoning.
Our findings reveal LCLMs' surprising ability to rival state-of-the-art retrieval and RAG systems, despite never having been explicitly trained for these tasks.
arXiv Detail & Related papers (2024-06-19T00:28:58Z) - User-LLM: Efficient LLM Contextualization with User Embeddings [24.099604517203606]
We propose User-LLM, a novel framework that leverages user embeddings to contextualize large language models (LLMs)
Our experiments on MovieLens, Amazon Review, and Google Local Review datasets demonstrate significant performance gains across various tasks.
arXiv Detail & Related papers (2024-02-21T08:03:27Z) - Persona-DB: Efficient Large Language Model Personalization for Response
Prediction with Collaborative Data Refinement [82.56964750522161]
We introduce Persona-DB, a simple framework consisting of a hierarchical construction process to improve generalization across task contexts.
In the task of response forecasting, Persona-DB demonstrates superior efficiency in maintaining accuracy with a significantly reduced retrieval size.
Our experiments also indicate a marked improvement of over 15% under cold-start scenarios, when users have extremely sparse data.
arXiv Detail & Related papers (2024-02-16T20:20:43Z) - Active Preference Inference using Language Models and Probabilistic Reasoning [13.523369679010685]
We introduce an inference-time algorithm that helps large language models infer user preferences.
Our algorithm uses a probabilistic model whose conditional distributions are defined by prompting an LLM.
Results in a simplified interactive web shopping setting with real product items show that an LLM equipped with our entropy reduction algorithm outperforms baselines.
arXiv Detail & Related papers (2023-12-19T09:58:54Z) - Adapting LLMs for Efficient, Personalized Information Retrieval: Methods
and Implications [0.7832189413179361]
Large Language Models (LLMs) excel in comprehending and generating human-like text.
This paper explores strategies for integrating Language Models (LLMs) with Information Retrieval (IR) systems.
arXiv Detail & Related papers (2023-11-21T02:01:01Z) - Query-Dependent Prompt Evaluation and Optimization with Offline Inverse
RL [62.824464372594576]
We aim to enhance arithmetic reasoning ability of Large Language Models (LLMs) through zero-shot prompt optimization.
We identify a previously overlooked objective of query dependency in such optimization.
We introduce Prompt-OIRL, which harnesses offline inverse reinforcement learning to draw insights from offline prompting demonstration data.
arXiv Detail & Related papers (2023-09-13T01:12:52Z) - OverPrompt: Enhancing ChatGPT through Efficient In-Context Learning [49.38867353135258]
We propose OverPrompt, leveraging the in-context learning capability of LLMs to handle multiple task inputs.
Our experiments show that OverPrompt can achieve cost-efficient zero-shot classification without causing significant detriment to task performance.
arXiv Detail & Related papers (2023-05-24T10:08:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.