Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesis

• URL: http://arxiv.org/abs/2311.12275v4
• Date: Tue, 16 Apr 2024 21:34:29 GMT
• Title: Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesis
• Abstract summary: We propose a novel framework to select and store the most representative data online in a self-supervised way. Our experiments show that the proposed framework achieves the best user-specific content-generating capability (accuracy) and fine-tuning speed (performance) compared with vanilla baselines.
• Score: 27.792167318819068
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: After a large language model (LLM) is deployed on edge devices, it is desirable for these devices to learn from user-generated conversation data to generate user-specific and personalized responses in real-time. However, user-generated data usually contains sensitive and private information, and uploading such data to the cloud for annotation is not preferred if not prohibited. While it is possible to obtain annotation locally by directly asking users to provide preferred responses, such annotations have to be sparse to not affect user experience. In addition, the storage of edge devices is usually too limited to enable large-scale fine-tuning with full user-generated data. It remains an open question how to enable on-device LLM personalization, considering sparse annotation and limited on-device storage. In this paper, we propose a novel framework to select and store the most representative data online in a self-supervised way. Such data has a small memory footprint and allows infrequent requests of user annotations for further fine-tuning. To enhance fine-tuning quality, multiple semantically similar pairs of question texts and expected responses are generated using the LLM. Our experiments show that the proposed framework achieves the best user-specific content-generating capability (accuracy) and fine-tuning speed (performance) compared with vanilla baselines. To the best of our knowledge, this is the very first on-device LLM personalization framework.

Related papers

• Sparse Personalized Text Generation with Multi-Trajectory Reasoning [95.19317487799292]
  PAT (Personalization with Aligned Trajectories) is a reasoning framework for cold-start LLM personalization.<n>It first retrieves information along two complementary trajectories: writing-style cues from stylistically similar users and topic-specific context from preference-aligned users.<n>It then employs a reinforcement learning-based, iterative dual-reasoning mechanism that enables the LLM to jointly refine and integrate these signals.
  arXiv  Detail & Related papers  (2026-04-27T20:58:49Z)
• Synthetic Interaction Data for Scalable Personalization in Large Language Models [67.31884245564086]
  We introduce a high-fidelity synthetic data generation framework called PersonaGym.<n>Unlike prior work that treats personalization as static persona-preference pairs, PersonaGym models a dynamic preference process.<n>We release PersonaAtlas, a large-scale, high-quality, and diverse synthetic dataset of high-fidelity multi-turn personalized interaction trajectories.
  arXiv  Detail & Related papers  (2026-02-12T20:41:22Z)
• Reasoning-Based Personalized Generation for Users with Sparse Data [120.94029850012045]
  We introduce GraSPer, a novel framework for enhancing personalized text generation under sparse context.<n>GraSPer first augments user context by predicting items that the user would likely interact with in the future.<n>With reasoning alignment, it then generates texts for these interactions to enrich the augmented context.<n>In the end, it generates personalized outputs conditioned on both the real and synthetic histories.
  arXiv  Detail & Related papers  (2026-01-31T01:54:23Z)
• Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences [73.5779077857545]
  We build a framework where a local model uses these instructions to rewrite queries, only hiding details deemed sensitive by the user, before sending them to an external model.<n>Experiments with lightweight local LLMs show that, after fine-tuning, they markedly exceed the performance of much larger zero-shot models.<n>At the same time, the system still faces challenges in fully adhering to user instructions, underscoring the need for models with a better understanding of user-defined privacy preferences.
  arXiv  Detail & Related papers  (2025-07-07T18:22:55Z)
• CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steering [68.91862701376155]
  CoSteer is a novel collaborative framework that enables decoding-time personalization through localized delta steering.<n>We formulate token-level optimization as an online learning problem, where local delta vectors dynamically adjust the remote LLM's logits.<n>This approach preserves privacy by transmitting only the final steered tokens rather than raw data or intermediate vectors.
  arXiv  Detail & Related papers  (2025-07-07T08:32:29Z)
• Embedding-to-Prefix: Parameter-Efficient Personalization for Pre-Trained Large Language Models [6.445337954429245]
  Large language models (LLMs) excel at generating contextually relevant content.<n>We propose Embedding-to-Prefix (E2P), a parameter-efficient method that injects context embeddings into an LLM's hidden representation space.<n>We evaluate E2P across two public datasets and in a production setting: dialogue personalization on Persona-Chat, contextual headline generation on PENS, and large-scale personalization for music and podcast consumption.
  arXiv  Detail & Related papers  (2025-05-16T13:34:25Z)
• Multi-agents based User Values Mining for Recommendation [52.26100802380767]
  We propose a zero-shot multi-LLM collaborative framework for effective and accurate user value extraction.<n>We apply text summarization techniques to condense item content while preserving essential meaning.<n>To mitigate hallucinations, we introduce two specialized agent roles: evaluators and supervisors.
  arXiv  Detail & Related papers  (2025-05-02T04:01:31Z)
• Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection [5.174560360759384]
  Personalization of Large Language Models (LLMs) is important in practical applications to accommodate the individual needs of different mobile users. We present XPerT, a new technique that ensure proper selection of such already personalized LLMs based on explainability about how they were being fine-tuned. Experiment results show that XPerT reduces the costs of on-device LLM personalization by 83%, and improves its data efficiency by 51%.
  arXiv  Detail & Related papers  (2025-04-15T17:38:06Z)
• Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalization [68.79814761867314]
  We propose Difference-aware Personalization Learning (DPL) to enhance Large Language Models (LLMs) personalization. DPL strategically selects representative users for comparison and establishes a structured standard to extract task-relevant differences. Experiments on real-world datasets demonstrate that DPL significantly enhances LLM personalization.
  arXiv  Detail & Related papers  (2025-03-04T09:53:26Z)
• Personalized Language Model Learning on Text Data Without User Identifiers [79.36212347601223]
  We propose to let each mobile device maintain a user-specific distribution to dynamically generate user embeddings. To prevent the cloud from tracking users via uploaded embeddings, the local distributions of different users should either be derived from a linearly dependent space. Evaluation on both public and industrial datasets reveals a remarkable improvement in accuracy from incorporating anonymous user embeddings.
  arXiv  Detail & Related papers  (2025-01-10T15:46:19Z)
• Optimizing Data Delivery: Insights from User Preferences on Visuals, Tables, and Text [59.68239795065175]
  We conduct a user study where users are shown a question and asked what they would prefer to see. We use the data to establish that a user's personal traits does influence the data outputs that they prefer.
  arXiv  Detail & Related papers  (2024-11-12T00:24:31Z)
• PersonalLLM: Tailoring LLMs to Individual Preferences [11.717169516971856]
  We present a public benchmark, PersonalLLM, focusing on adapting LLMs to provide maximal benefits for a particular user. We curate open-ended prompts paired with many high-quality answers over which users would be expected to display heterogeneous latent preferences. Our dataset and generated personalities offer an innovative testbed for developing personalization algorithms.
  arXiv  Detail & Related papers  (2024-09-30T13:55:42Z)
• Scalable Dynamic Embedding Size Search for Streaming Recommendation [54.28404337601801]
  Real-world recommender systems often operate in streaming recommendation scenarios. Number of users and items continues to grow, leading to substantial storage resource consumption. We learn Lightweight Embeddings for streaming recommendation, called SCALL, which can adaptively adjust the embedding sizes of users/items.
  arXiv  Detail & Related papers  (2024-07-22T06:37:24Z)
• Crayon: Customized On-Device LLM via Instant Adapter Blending and Edge-Server Hybrid Inference [20.666893617591136]
  We propose Crayon, a novel approach for on-device LLM customization. We develop a device-server hybrid inference strategy, which deftly allocates more demanding queries or non-customized tasks to a larger, more capable LLM on a server.
  arXiv  Detail & Related papers  (2024-06-11T07:00:08Z)
• Aligning Large Language Models with Self-generated Preference Data [72.99676237703099]
  We propose a new framework that boosts the alignment of large language models (LLMs) with human preferences. Our key idea is leveraging the human prior knowledge within the small (seed) data. We introduce a noise-aware preference learning algorithm to mitigate the risk of low quality within generated preference data.
  arXiv  Detail & Related papers  (2024-06-06T18:01:02Z)
• Partial Federated Learning [26.357723187375665]
  Federated Learning (FL) is a popular algorithm to train machine learning models on user data constrained to edge devices. We propose a new algorithm called Partial Federated Learning (PartialFL), where a machine learning model is trained using data where a subset of data modalities can be made available to the server.
  arXiv  Detail & Related papers  (2024-03-03T21:04:36Z)
• Modeling Dynamic User Preference via Dictionary Learning for Sequential Recommendation [133.8758914874593]
  Capturing the dynamics in user preference is crucial to better predict user future behaviors because user preferences often drift over time. Many existing recommendation algorithms -- including both shallow and deep ones -- often model such dynamics independently. This paper considers the problem of embedding a user's sequential behavior into the latent space of user preferences.
  arXiv  Detail & Related papers  (2022-04-02T03:23:46Z)
• SemiPFL: Personalized Semi-Supervised Federated Learning Framework for Edge Intelligence [15.590672649077817]
  This paper proposes a novel semi-supervised federated learning (SemiPFL) framework to support edge users having no label or limited labeled datasets. In this work, edge users collaborate to train a hyper-network in the server, generating personalized autoencoders for each user. After receiving updates from edge users, the server produces a set of base models for each user, which the users locally aggregate them using their own labeled dataset.
  arXiv  Detail & Related papers  (2022-03-15T18:09:15Z)
• Generating private data with user customization [9.415164800448853]
  Mobile devices can produce and store large amounts of data that can enhance machine learning models. However, this data may contain private information specific to the data owner that prevents the release of the data. We want to reduce the correlation between user-specific private information and the data while retaining the useful information.
  arXiv  Detail & Related papers  (2020-12-02T19:13:58Z)
• Federated Learning of User Authentication Models [69.93965074814292]
  We propose Federated User Authentication (FedUA), a framework for privacy-preserving training of machine learning models. FedUA adopts federated learning framework to enable a group of users to jointly train a model without sharing the raw inputs. We show our method is privacy-preserving, scalable with number of users, and allows new users to be added to training without changing the output layer.
  arXiv  Detail & Related papers  (2020-07-09T08:04:38Z)
• Unsupervised Model Personalization while Preserving Privacy and Scalability: An Open Problem [55.21502268698577]
  This work investigates the task of unsupervised model personalization, adapted to continually evolving, unlabeled local user images. We provide a novel Dual User-Adaptation framework (DUA) to explore the problem. This framework flexibly disentangles user-adaptation into model personalization on the server and local data regularization on the user device.
  arXiv  Detail & Related papers  (2020-03-30T09:35:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.