Personalize Your LLM: Fake it then Align it

• URL: http://arxiv.org/abs/2503.01048v3
• Date: Wed, 05 Mar 2025 18:59:19 GMT
• Title: Personalize Your LLM: Fake it then Align it
• Authors: Yijing Zhang, Dyah Adila, Changho Shin, Frederic Sala,
• Abstract summary: CHAMELEON is a scalable and efficient personalization approach that uses self-generated personal preference data and representation editing.<n>Our experiments show that CHAMELEON efficiently adapts models to personal preferences, improving instruction-tuned models and outperforms two personalization baselines by an average of 40%.
• Score: 12.436528089142698
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalizing large language models (LLMs) is essential for delivering tailored interactions that improve user experience. Many existing personalization methods require fine-tuning LLMs for each user, rendering them prohibitively expensive for widespread adoption. Although retrieval-based approaches offer a more compute-efficient alternative, they still depend on large, high-quality datasets that are not consistently available for all users. To address this challenge, we propose CHAMELEON, a scalable and efficient personalization approach that uses (1) self-generated personal preference data and (2) representation editing to enable quick and cost-effective personalization. Our experiments on various tasks, including those from the LaMP personalization benchmark, show that CHAMELEON efficiently adapts models to personal preferences, improving instruction-tuned models and outperforms two personalization baselines by an average of 40% across two model architectures.

Related papers

• NextQuill: Causal Preference Modeling for Enhancing LLM Personalization [82.15961484963256]
  We introduce NextQuill, a novel personalization framework grounded in causal preference modeling.<n>Building on this insight, NextQuill introduces two complementary alignment strategies.<n> Experiments across multiple personalization benchmarks demonstrate that NextQuill significantly improves personalization quality.
  arXiv  Detail & Related papers  (2025-06-03T02:08:55Z)
• Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Models [6.445337954429245]
  Large language models (LLMs) excel at generating contextually relevant content.<n>We propose Embedding-to-Prefix (E2P), a parameter-efficient method that injects context embeddings into an LLM's hidden representation space.<n>We evaluate E2P across two public datasets and in a production setting: dialogue personalization on Persona-Chat, contextual headline generation on PENS, and large-scale personalization for music and podcast consumption.
  arXiv  Detail & Related papers  (2025-05-16T13:34:25Z)
• HyPerAlign: Hypotheses-driven Personalized Alignment [24.67727411391369]
  We propose a hypotheses-driven personalization approach (HyPerAlign) for large language models (LLMs) For deliberative alignment, the helpfulness of LLM models is improved by up to $70%$ on average. For authorship attribution, results indicate consistently high win-rates (commonly $>90%$) against state-of-the-art preference fine-tuning approaches.
  arXiv  Detail & Related papers  (2025-04-29T18:01:46Z)
• Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection [5.174560360759384]
  Personalization of Large Language Models (LLMs) is important in practical applications to accommodate the individual needs of different mobile users. We present XPerT, a new technique that ensure proper selection of such already personalized LLMs based on explainability about how they were being fine-tuned. Experiment results show that XPerT reduces the costs of on-device LLM personalization by 83%, and improves its data efficiency by 51%.
  arXiv  Detail & Related papers  (2025-04-15T17:38:06Z)
• Personalized Language Models via Privacy-Preserving Evolutionary Model Merging [57.161917758405465]
  Personalization in large language models (LLMs) seeks to tailor models to individual user or user group preferences. We propose Privacy-Preserving Model Merging via Evolutionary Algorithms (PriME) PriME employs gradient-free methods to directly optimize task-specific metrics while preserving user privacy.
  arXiv  Detail & Related papers  (2025-03-23T09:46:07Z)
• Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalization [68.79814761867314]
  We propose Difference-aware Personalization Learning (DPL) to enhance Large Language Models (LLMs) personalization.<n>DPL strategically selects representative users for comparison and establishes a structured standard to extract task-relevant differences.<n>Experiments on real-world datasets demonstrate that DPL significantly enhances LLM personalization.
  arXiv  Detail & Related papers  (2025-03-04T09:53:26Z)
• FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users [111.56469697145519]
  We propose Few-Shot Preference Optimization, which reframes reward modeling as a meta-learning problem.<n>Under this framework, an LLM learns to quickly adapt to a user via a few labeled preferences from that user, constructing a personalized reward function for them.<n>We generate over 1M synthetic personalized preferences using publicly available LLMs.<n>We evaluate FSPO on personalized open-ended generation for up to 1,500 synthetic users across three domains: movie reviews, pedagogical adaptation based on educational background, and general question answering, along with a controlled human study.
  arXiv  Detail & Related papers  (2025-02-26T17:08:46Z)
• Personalized Preference Fine-tuning of Diffusion Models [75.22218338096316]
  We introduce PPD, a multi-reward optimization objective that aligns diffusion models with personalized preferences. With PPD, a diffusion model learns the individual preferences of a population of users in a few-shot way. Our approach achieves an average win rate of 76% over Stable Cascade, generating images that more accurately reflect specific user preferences.
  arXiv  Detail & Related papers  (2025-01-11T22:38:41Z)
• ULMRec: User-centric Large Language Model for Sequential Recommendation [16.494996929730927]
  We propose ULMRec, a framework that integrates user personalized preferences into Large Language Models.<n>Extensive experiments on two public datasets demonstrate that ULMRec significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2024-12-07T05:37:00Z)
• LLMs + Persona-Plug = Personalized LLMs [41.60364110693824]
  Personalization plays a critical role in numerous language tasks and applications, since users with the same requirements may prefer diverse outputs based on their individual interests. This has led to the development of various personalized approaches aimed at adapting large language models (LLMs) to generate customized outputs aligned with user preferences. We propose a novel personalized LLM model, ours. It constructs a user-specific embedding for each individual by modeling all her historical contexts through a lightweight plug-in user embedder module.
  arXiv  Detail & Related papers  (2024-09-18T11:54:45Z)
• Comparing Retrieval-Augmentation and Parameter-Efficient Fine-Tuning for Privacy-Preserving Personalization of Large Language Models [21.115495457454365]
  This paper presents the first systematic comparison between two approaches on a wide range of personalization tasks. We find that RAG-based and PEFT-based personalization methods on average yield 14.92% and 1.07% improvements over the non-personalized LLM, respectively.
  arXiv  Detail & Related papers  (2024-09-14T19:18:26Z)
• Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation [50.837277466987345]
  We focus on the field of large language models (LLMs) for recommendation. We propose RecLoRA, which incorporates a Personalized LoRA module that maintains independent LoRAs for different users. We also design a Few2Many Learning Strategy, using a conventional recommendation model as a lens to magnify small training spaces to full spaces.
  arXiv  Detail & Related papers  (2024-08-07T04:20:28Z)
• PEFT-U: Parameter-Efficient Fine-Tuning for User Personalization [9.594958534074074]
  We introduce the PEFT-U Benchmark: a new dataset for building and evaluating NLP models for user personalization. We explore the challenge of efficiently personalizing LLMs to accommodate user-specific preferences in the context of diverse user-centered tasks.
  arXiv  Detail & Related papers  (2024-07-25T14:36:18Z)
• Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond [87.1712108247199]
  Our goal is to establish a Unified paradigm for Multi-modal Personalization systems (UniMP) We develop a generic and personalization generative framework, that can handle a wide range of personalized needs. Our methodology enhances the capabilities of foundational language models for personalized tasks.
  arXiv  Detail & Related papers  (2024-03-15T20:21:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.