Personalized Language Models via Privacy-Preserving Evolutionary Model Merging

• URL: http://arxiv.org/abs/2503.18008v1
• Date: Sun, 23 Mar 2025 09:46:07 GMT
• Title: Personalized Language Models via Privacy-Preserving Evolutionary Model Merging
• Authors: Kyuyoung Kim, Jinwoo Shin, Jaehyung Kim,
• Abstract summary: Personalization in large language models (LLMs) seeks to tailor models to individual user or user group preferences.<n>We propose Privacy-Preserving Model Merging via Evolutionary Algorithms (PriME)<n>PriME employs gradient-free methods to directly optimize task-specific metrics while preserving user privacy.
• Score: 57.161917758405465
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Personalization in large language models (LLMs) seeks to tailor models to individual user or user group preferences. Prompt-based methods augment queries with user preference information, whereas training-based methods directly encode preferences into model parameters for more effective personalization. Despite achieving some success in personalizing LLMs, prior methods often fail to directly optimize task-specific metrics and lack explicit privacy-preservation mechanisms. To address these limitations, we propose Privacy-Preserving Model Merging via Evolutionary Algorithms (PriME), a novel approach to personalization that employs gradient-free methods to directly optimize task-specific metrics while preserving user privacy. By incorporating privacy preservation into optimization, PriME produces a personalized module that effectively captures the target user's preferences while minimizing the privacy risks for the users sharing their private information. Experiments on the LaMP benchmark show that PriME outperforms both prompt-based and training-based methods, achieving up to a 45% performance improvement over the prior art. Further analysis shows that PriME achieves a significantly better privacy-utility trade-off, highlighting the potential of evolutionary approaches for privacy-preserving LLM personalization.

Related papers

• Adaptive Clipping for Privacy-Preserving Few-Shot Learning: Enhancing Generalization with Limited Data [12.614480013684759]
  We introduce a novel approach called Meta-Clip to enhance the utility of privacy-preserving few-shot learning methods. By dynamically adjusting clipping thresholds during the training process, our Adaptive Clipping method provides fine-grained control over the disclosure of sensitive information. We demonstrate the effectiveness of our approach in minimizing utility degradation, showcasing a superior privacy-preserving trade-off compared to existing privacy-preserving techniques.
  arXiv  Detail & Related papers  (2025-03-27T05:14:18Z)
• Multi-Objective Optimization for Privacy-Utility Balance in Differentially Private Federated Learning [12.278668095136098]
  Federated learning (FL) enables collaborative model training across distributed clients without sharing raw data. We propose an adaptive clipping mechanism that dynamically adjusts the clipping norm using a multi-objective optimization framework. Our results show that adaptive clipping consistently outperforms fixed-clipping baselines, achieving improved accuracy under the same privacy constraints.
  arXiv  Detail & Related papers  (2025-03-27T04:57:05Z)
• Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalization [68.79814761867314]
  We propose Difference-aware Personalization Learning (DPL) to enhance Large Language Models (LLMs) personalization.<n>DPL strategically selects representative users for comparison and establishes a structured standard to extract task-relevant differences.<n>Experiments on real-world datasets demonstrate that DPL significantly enhances LLM personalization.
  arXiv  Detail & Related papers  (2025-03-04T09:53:26Z)
• Advancing Personalized Federated Learning: Integrative Approaches with AI for Enhanced Privacy and Customization [0.0]
  This paper proposes a novel approach that enhances PFL with cutting-edge AI techniques.<n>We present a model that boosts the performance of individual client models and ensures robust privacy-preserving mechanisms.<n>This work paves the way for a new era of truly personalized and privacy-conscious AI systems.
  arXiv  Detail & Related papers  (2025-01-30T07:03:29Z)
• Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes [50.544186914115045]
  Large language models (LLMs) are increasingly embedded in everyday applications. Ensuring their alignment with the diverse preferences of individual users has become a critical challenge. We present a novel framework for few-shot steerable alignment.
  arXiv  Detail & Related papers  (2024-12-18T16:14:59Z)
• Guided Profile Generation Improves Personalization with LLMs [3.2685922749445617]
  In modern commercial systems, including Recommendation, Ranking, and E-Commerce platforms, there is a trend towards incorporating Personalization context as input into Large Language Models (LLMs) We propose Guided Profile Generation (GPG), a general method designed to generate personal profiles in natural language. Our experimental results show that GPG improves LLM's personalization ability across different tasks, for example, it increases 37% accuracy in predicting personal preference compared to directly feeding the LLMs with raw personal context.
  arXiv  Detail & Related papers  (2024-09-19T21:29:56Z)
• Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment [8.028743532294532]
  We introduce a novel reference-based attack framework specifically for analyzing preference data called PREMIA. We provide empirical evidence that DPO models are more vulnerable to MIA compared to PPO models.
  arXiv  Detail & Related papers  (2024-07-08T22:53:23Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• Differentially Private Fine-Tuning of Diffusion Models [22.454127503937883]
  The integration of Differential Privacy with diffusion models (DMs) presents a promising yet challenging frontier. Recent developments in this field have highlighted the potential for generating high-quality synthetic data by pre-training on public data. We propose a strategy optimized for private diffusion models, which minimizes the number of trainable parameters to enhance the privacy-utility trade-off.
  arXiv  Detail & Related papers  (2024-06-03T14:18:04Z)
• Differentially Private Deep Model-Based Reinforcement Learning [47.651861502104715]
  We introduce PriMORL, a model-based RL algorithm with formal differential privacy guarantees. PriMORL learns an ensemble of trajectory-level DP models of the environment from offline data.
  arXiv  Detail & Related papers  (2024-02-08T10:05:11Z)
• Private Set Generation with Discriminative Information [63.851085173614]
  Differentially private data generation is a promising solution to the data privacy challenge. Existing private generative models are struggling with the utility of synthetic samples. We introduce a simple yet effective method that greatly improves the sample utility of state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-07T10:02:55Z)
• Group privacy for personalized federated learning [4.30484058393522]
  Federated learning is a type of collaborative machine learning, where participating clients process their data locally, sharing only updates to the collaborative model. We propose a method to provide group privacy guarantees exploiting some key properties of $d$-privacy.
  arXiv  Detail & Related papers  (2022-06-07T15:43:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.