Weightless Fine-Tuning: Personalizing LLMs via Logit-Space Transport

• URL: http://arxiv.org/abs/2608.11342v1
• Date: Tue, 11 Aug 2026 18:49:03 GMT
• Title: Weightless Fine-Tuning: Personalizing LLMs via Logit-Space Transport
• Abstract summary: Weightless Fine-Tuning (WFT) is a training-free decoding-time method that approximates the distributional effect of supervised adaptation without updates.<n>WFT achieves the best average performance across datasets, matches or exceeds SFT on individual tasks, and outperforms other lightweight baselines on average.
• Score: 35.12591387008241
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Supervised fine-tuning (SFT) is a standard approach for adapting LLMs to a target distribution, but in settings such as personalization, where each author requires separate weight access, optimization, storage, and retraining, its costs become prohibitive. We propose Weightless Fine-Tuning (WFT), a training-free decoding-time method that approximates the distributional effect of SFT without weight updates. WFT computes supervised residuals on an author's training sequence and transports them to the current prompt through a cross-prefix transport operator estimated from dropout-induced cross-covariance. The operator captures how a perturbation at one context propagates to predictions at another, replacing gradient-based parameter updates with logit-space corrections. On three LaMP personalization benchmarks, WFT achieves the best average performance across datasets, matches or exceeds SFT on individual tasks, and outperforms other lightweight baselines on average. In a budget-controlled comparison, WFT approaches SFT performance using less than 7% of the effective computation. Logit-level analysis shows a cosine similarity of 0.875 between the logit shifts induced by WFT and SFT over 95% of the next-token probability mass, suggesting that WFT captures the distributional effect of supervised adaptation without modifying model weights.

Related papers

• Online Self-Weighted Fine-Tuning [13.544097764061886]
  Reinforcement learning (RL) based methods adapt update strength using model-generated rollouts.<n>We propose textbfOnline Self-Weighted Fine-Tuning (OSW-FT), a simple method that augments SFT with online, trajectory-level weighting.<n>OSW-FT consistently improves over SFT on small-to-medium scale models.
  arXiv  Detail & Related papers  (2026-09-01T05:12:45Z)
• PreFT: Prefill-only finetuning for efficient inference [54.58291801311547]
  We develop and release an efficient implementation of two prefill-only PEFTs, LoRA and ReFT, on the vLLM inference engine.<n>We show that serving multi-user PreFTs is more efficient than traditional PEFTs ($1.9times$ the throughput when serving $512$ adapters on Llama 3.1 70B)
  arXiv  Detail & Related papers  (2026-05-14T00:19:41Z)
• Preserving Foundational Capabilities in Flow-Matching VLAs through Conservative SFT [12.18095306054314]
  Conservative Supervised Fine-Tuning (ConSFT) is an optimization objective that adapts to target distributions while mitigating catastrophic forgetting.<n>We evaluate ConSFT on the LIBERO and RoboTwin benchmarks across state-of-the-art flow-matching VLAs.
  arXiv  Detail & Related papers  (2026-05-09T10:59:03Z)
• An Empirical Study of SFT-DPO Interaction and Parameterization in Small Language Models [1.7559669963238091]
  We compare SFT-only, DPO-only, and staged SFT-to-DPO training alongside full fine-tuning (FFT) versus LoRA on a GPT-2-scale decoder.<n>DPO yields small, task-dependent gains over strong SFT and can match competitive SFT accuracy without a warm start.
  arXiv  Detail & Related papers  (2026-03-20T16:27:03Z)
• Trust-Region Adaptive Policy Optimization [82.09255251747818]
  Post-training methods play an important role in improving large language models' (LLMs) complex reasoning abilities.<n>We introduce TRAPO, a framework that interleavesSupervised Fine-Tuning (SFT) and Reinforcement Learning (RL) within each training instance.<n>Experiments on five mathematical reasoning benchmarks show that TRAPO consistently surpasses standard SFT, RL, and SFT-then-RL pipelines.
  arXiv  Detail & Related papers  (2025-12-19T14:37:07Z)
• Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data [73.04828796123581]
  Supervised fine-tuning (SFT) has become a crucial step for aligning pretrained large language models (LLMs)<n>We introduce Discriminative Fine-Tuning (DFT), an improved variant of SFT, which mitigates the burden of collecting human-labeled preference data.<n>Our contributions include: (i) a discriminative probabilistic framework for fine-tuning LLMs by explicitly modeling the discriminative likelihood of an answer among all possible outputs given an input; (ii) efficient algorithms to optimize this discriminative likelihood; and (iii) extensive experiments demonstrating DFT's effectiveness
  arXiv  Detail & Related papers  (2025-02-25T22:38:55Z)
• UFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Function [18.54945183526789]
  We introduce Unified Fine-Tuning (UFT), which integrates SFT and alignment into a single training stage.<n>Our experimental results demonstrate that UFT outperforms SFT on instruction-tuning data alone.<n>When combining instruction-tuning data with alignment data, UFT effectively prevents catastrophic forgetting.
  arXiv  Detail & Related papers  (2024-10-28T18:34:25Z)
• Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process [19.986235452236272]
  Supervised Fine-Tuning (SFT) and Preference Optimization (PO) are key processes for aligning Language Models (LMs) with human preferences post pre-training.<n>We introduce Intuitive Fine-Tuning (IFT) to integrate SFT and PO into a single process.<n>IFT performs comparably or even superiorly to SFT and some typical PO methods across several tasks.
  arXiv  Detail & Related papers  (2024-05-20T08:23:28Z)
• From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers [52.199303258423306]
  We propose a novel density loss that encourages higher activation sparsity in pre-trained models. Our proposed method, textbfDEFT, can consistently reduce activation density by up to textbf44.94% on RoBERTa$_mathrmLarge$ and by textbf53.19% (encoder density) and textbf90.60% (decoder density) on Flan-T5$_mathrmXXL$.
  arXiv  Detail & Related papers  (2024-02-02T21:25:46Z)
• Sparse is Enough in Fine-tuning Pre-trained Large Language Models [98.46493578509039]
  We propose a gradient-based sparse fine-tuning algorithm, named Sparse Increment Fine-Tuning (SIFT) We validate its effectiveness on a range of tasks including the GLUE Benchmark and Instruction-tuning.
  arXiv  Detail & Related papers  (2023-12-19T06:06:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.