PLUME: Parameter-Efficient Personalization of Large Language Models via Low-Rank User Modulation in Shared Subspaces

• URL: http://arxiv.org/abs/2609.04715v1
• Date: Fri, 04 Sep 2026 04:38:50 GMT
• Title: PLUME: Parameter-Efficient Personalization of Large Language Models via Low-Rank User Modulation in Shared Subspaces
• Abstract summary: PLUME is a framework that achieves efficient and expressive per-user adaptation by leveraging a shared task-specific subspace.<n> PLUME achieves comparable or superior performance to strong baselines, while reducing per-user parameters by over 95%.
• Score: 10.890278280201592
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personalizing large language models (LLMs) is essential for delivering AI assistance that aligns with individual users' styles, intents, and preferences. While per-user fine-tuning can substantially enhance personalization quality, it introduces significant parameter and storage overhead, limiting scalability to large user populations. We propose PLUME (Personalized Low-Rank Adaptation through User Modulation and Shared Subspace), a lightweight framework that achieves efficient and expressive per-user adaptation by leveraging a shared task-specific subspace. Specifically, PLUME first learns a global task subspace from aggregated user data. Personalization is then achieved by training only a lightweight small square matrix within this subspace, enabling each user to obtain a tailored model while keeping shared components fixed. Cross-layer shared parameters and rank-1 residual terms are further introduced to significantly reduce redundancy while maintaining expressiveness. Experiments on multiple personalized text generation benchmarks demonstrate that PLUME achieves comparable or superior performance to strong baselines, while reducing per-user parameters by over 95%. These results establish shared-subspace modulation with minimal residuals as a scalable and semantically grounded approach to LLM personalization.

Related papers

• Aplaud: Adaptive Personalized Low-Rank Decomposition for User-Specific LLM [12.162804028842586]
  We study the problem of personalized survey response prediction using fine-tuned large language models (LLMs)<n>This task poses unique challenges: limited per-user training data, scalability of model storage, and the need to exploit shared structure across survey questions.<n>We propose Aplaud, a lightweight and scalable framework for LLM personalization.
  arXiv  Detail & Related papers  (2026-09-04T05:09:17Z)
• Beyond Retrieval: Learning Compact User Representations for Scalable LLM Personalization [9.496320762102252]
  We propose TAP-PER, a framework that encodes user preferences as learnable representations.<n>Experiments on six LaMP tasks show that TAP-PER consistently outperforms prompt-based and model-based baselines.<n> TAP-PER uses 130x fewer per-user parameters than OPPU and roughly half the total parameter footprint of PER-PCS at the 1,000-user scale.
  arXiv  Detail & Related papers  (2026-06-03T07:32:18Z)
• CLIPer: Tailoring Diverse User Preference via Classifier-Guided Inference-Time Personalization [28.672568353736335]
  We introduce textbfCLIPer(textbfClassifier-guided textbfInference-time textbfPersonalization), a lightweight personalization approach.<n>Our method eliminates the need for extensive fine-tuning, inducing negligible additional computational overhead.
  arXiv  Detail & Related papers  (2026-05-08T02:47:30Z)
• Synthetic Interaction Data for Scalable Personalization in Large Language Models [67.31884245564086]
  We introduce a high-fidelity synthetic data generation framework called PersonaGym.<n>Unlike prior work that treats personalization as static persona-preference pairs, PersonaGym models a dynamic preference process.<n>We release PersonaAtlas, a large-scale, high-quality, and diverse synthetic dataset of high-fidelity multi-turn personalized interaction trajectories.
  arXiv  Detail & Related papers  (2026-02-12T20:41:22Z)
• Instant Personalized Large Language Model Adaptation via Hypernetwork [56.512539596908745]
  Profile-to-PEFT is a scalable framework that employs a hypernetwork, trained end-to-end to map a user's encoded profile directly to a full set of adapter parameters.<n>We show that our method outperforms both prompt-based personalization and OPPU while using substantially fewer computational resources at deployment.
  arXiv  Detail & Related papers  (2025-10-18T00:41:25Z)
• LoRe: Personalizing LLMs via Low-Rank Reward Modeling [47.12507639759984]
  We introduce a novel framework that leverages low-rank preference modeling to efficiently learn and generalize user-specific reward functions.<n>We validate our method on multiple preference datasets, demonstrating superior generalization to unseen users and improved accuracy in preference prediction tasks.
  arXiv  Detail & Related papers  (2025-04-20T01:16:24Z)
• Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalization [68.79814761867314]
  We propose Difference-aware Personalization Learning (DPL) to enhance Large Language Models (LLMs) personalization.<n>DPL strategically selects representative users for comparison and establishes a structured standard to extract task-relevant differences.<n>Experiments on real-world datasets demonstrate that DPL significantly enhances LLM personalization.
  arXiv  Detail & Related papers  (2025-03-04T09:53:26Z)
• Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation [50.837277466987345]
  We focus on the field of large language models (LLMs) for recommendation. We propose RecLoRA, which incorporates a Personalized LoRA module that maintains independent LoRAs for different users. We also design a Few2Many Learning Strategy, using a conventional recommendation model as a lens to magnify small training spaces to full spaces.
  arXiv  Detail & Related papers  (2024-08-07T04:20:28Z)
• Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging [148.77027765872006]
  We study Reinforcement Learning from Personalized Human Feedback (RLPHF) problem. LLMs are aligned to multiple preferences by modeling alignment as a Multi-Objective Reinforcement Learning (MORL) problem. We show that we can achieve personalized alignment by decomposing preferences into multiple dimensions.
  arXiv  Detail & Related papers  (2023-10-17T20:22:13Z)
• Sample-Efficient Personalization: Modeling User Parameters as Low Rank Plus Sparse Components [30.32486162748558]
  Personalization of machine learning (ML) predictions for individual users/domains/enterprises is critical for practical recommendation systems. We propose a novel meta-learning style approach that models network weights as a sum of low-rank and sparse components. We show that AMHT-LRS solves the problem efficiently with nearly optimal sample complexity.
  arXiv  Detail & Related papers  (2022-10-07T12:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.