Democratizing Large Language Models via Personalized Parameter-Efficient
Fine-tuning
- URL: http://arxiv.org/abs/2402.04401v1
- Date: Tue, 6 Feb 2024 21:03:52 GMT
- Title: Democratizing Large Language Models via Personalized Parameter-Efficient
Fine-tuning
- Authors: Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, Meng
Jiang
- Abstract summary: Personalization in large language models (LLMs) is increasingly important.
One PEFT Per User (OPPU) employs personalized parameter-efficient fine-tuning (PEFT) modules to store user-specific behavior patterns and preferences.
OPPU significantly outperforms existing prompt-based methods across seven diverse tasks in the LaMP benchmark.
- Score: 39.0799671550279
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Personalization in large language models (LLMs) is increasingly important,
aiming to align LLM's interactions, content, and recommendations with
individual user preferences. Recent advances in LLM personalization have
spotlighted effective prompt design, by enriching user queries with
non-parametric knowledge through behavior history retrieval and textual
profiles. However, these approaches were limited due to a lack of model
ownership, resulting in constrained customization and privacy issues. Moreover,
they often failed to accurately capture user behavior patterns, especially in
cases where user data were complex and dynamic. To address these shortcomings,
we introduce One PEFT Per User (OPPU), which employs personalized
parameter-efficient fine-tuning (PEFT) modules, to store user-specific behavior
patterns and preferences. By plugging in users' personal PEFT parameters, they
can own and use their LLMs personally. OPPU integrates parametric user
knowledge in the personal PEFT parameters with the non-parametric knowledge
acquired through retrieval and profile. This integration adapts individual LLMs
to user behavior shifts. Experimental results demonstrate that OPPU
significantly outperforms existing prompt-based methods across seven diverse
tasks in the LaMP benchmark. Further in-depth studies reveal OPPU's enhanced
capabilities in handling user behavior shifts, modeling users at different
active levels, maintaining robustness across various user history formats, and
displaying versatility with different PEFT methods.
Related papers
- PEFT-U: Parameter-Efficient Fine-Tuning for User Personalization [9.594958534074074]
We introduce the PEFT-U Benchmark: a new dataset for building and evaluating NLP models for user personalization.
We explore the challenge of efficiently personalizing LLMs to accommodate user-specific preferences in the context of diverse user-centered tasks.
arXiv Detail & Related papers (2024-07-25T14:36:18Z) - SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning [70.21358720599821]
Large language models (LLMs) hold the promise of solving diverse tasks when provided with appropriate natural language prompts.
We propose SELF-GUIDE, a multi-stage mechanism in which we synthesize task-specific input-output pairs from the student LLM.
We report an absolute improvement of approximately 15% for classification tasks and 18% for generation tasks in the benchmark's metrics.
arXiv Detail & Related papers (2024-07-16T04:41:58Z) - Few-shot Personalization of LLMs with Mis-aligned Responses [40.0349773257245]
This paper proposes a new approach for a few-shot personalization of large language models (LLMs)
Our key idea is to learn a set of personalized prompts for each user by progressively improving the prompts using LLMs.
During an iterative process of prompt improvement, we incorporate the contexts of mis-aligned responses by LLMs.
arXiv Detail & Related papers (2024-06-26T18:29:12Z) - Step-Back Profiling: Distilling User History for Personalized Scientific Writing [50.481041470669766]
Large language models (LLM) excel at a variety of natural language processing tasks, yet they struggle to generate personalized content for individuals.
We introduce STEP-BACK PROFILING to personalize LLMs by distilling user history into concise profiles.
Our approach outperforms the baselines by up to 3.6 points on the general personalization benchmark.
arXiv Detail & Related papers (2024-06-20T12:58:26Z) - HYDRA: Model Factorization Framework for Black-Box LLM Personalization [36.21602686505842]
Personalization has emerged as a critical research area in modern intelligent systems.
Despite the remarkable few-shot capabilities exhibited by black-box large language models (LLMs), the inherent opacity of their model parameters presents significant challenges in aligning the generated output with individual expectations.
We propose HYDRA, a model factorization framework that captures both user-specific behavior patterns from historical data and shared general knowledge among all users to deliver personalized generation.
arXiv Detail & Related papers (2024-06-05T03:08:46Z) - Knowledge Graph Tuning: Real-time Large Language Model Personalization based on Human Feedback [5.778012023739487]
We propose Knowledge Graph Tuning (KGT) to personalize large language models (LLMs)
KGT extracts personalized factual knowledge triples from users' queries and feedback and optimize KGs without modifying the LLM parameters.
Experiments with state-of-the-art LLMs, including GPT-2, Llama2, and Llama3, show that KGT significantly improves personalization performance while reducing latency and GPU memory costs.
arXiv Detail & Related papers (2024-05-30T04:57:03Z) - Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts [95.09994361995389]
Relative Preference Optimization (RPO) is designed to discern between more and less preferred responses derived from both identical and related prompts.
RPO has demonstrated a superior ability to align large language models with user preferences and to improve their adaptability during the training process.
arXiv Detail & Related papers (2024-02-12T22:47:57Z) - Integrating Summarization and Retrieval for Enhanced Personalization via
Large Language Models [11.950478880423733]
Personalization is an essential factor in user experience with natural language processing (NLP) systems.
With the emergence of Large Language Models (LLMs), a key question is how to leverage these models to better personalize user experiences.
We propose a novel summary-augmented personalization with task-aware user summaries generated by LLMs.
arXiv Detail & Related papers (2023-10-30T23:40:41Z) - Interactive Hyperparameter Optimization in Multi-Objective Problems via
Preference Learning [65.51668094117802]
We propose a human-centered interactive HPO approach tailored towards multi-objective machine learning (ML)
Instead of relying on the user guessing the most suitable indicator for their needs, our approach automatically learns an appropriate indicator.
arXiv Detail & Related papers (2023-09-07T09:22:05Z) - The Minority Matters: A Diversity-Promoting Collaborative Metric
Learning Algorithm [154.47590401735323]
Collaborative Metric Learning (CML) has recently emerged as a popular method in recommendation systems.
This paper focuses on a challenging scenario where a user has multiple categories of interests.
We propose a novel method called textitDiversity-Promoting Collaborative Metric Learning (DPCML)
arXiv Detail & Related papers (2022-09-30T08:02:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.