Cautious Context Steering for Language Model Personalization

• URL: http://arxiv.org/abs/2608.05813v1
• Date: Thu, 06 Aug 2026 09:45:25 GMT
• Title: Cautious Context Steering for Language Model Personalization
• Abstract summary: Existing methods typically train a separate adapter for each user or learn a reward model whose scores depend on the user.<n>We propose Cautious Context Steering, which adds a lightweight adapter to a frozen backbone LM to decide at each token whether and how strongly user context should affect generation.<n>A single CCS adapter trained on only one dataset improves generation quality both in-domain and across four out-of-distribution personalization benchmarks.
• Score: 12.733264310259836
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Personalizing language models (LMs) to individual user preferences is essential for aligning responses with diverse goals and backgrounds. Existing methods typically train a separate adapter for each user or learn a reward model whose scores depend on the user. Despite explicitly optimizing for each user, these methods must learn from limited observations and therefore suffer from data sparsity and poor generalization to unseen users and domains. In-context learning (ICL) and Context Steering (CoS) can instead provide more effective personalization by conditioning the base LM directly on user context and leveraging its pretrained capabilities without per-user training. Yet neither adapts the influence of that context across decoding steps: ICL leaves it uncontrolled, whereas CoS applies a fixed steering coefficient and requires two LM forward passes per step. We propose Cautious Context Steering (CCS), which adds a lightweight adapter to a frozen backbone LM to decide at each token whether and how strongly user context should affect generation. The adapter learns this behavior from an oracle context-conditioned LM and preserves the base LM when the context is not helpful. A single CCS adapter trained on only one dataset improves generation quality both in-domain and across four out-of-distribution personalization benchmarks, demonstrating robust generalization to new users and domains. CCS also avoids per-user fine-tuning and the additional context-conditioned forward pass required by CoS, substantially reducing inference cost.

Related papers

• Learning Transferable Latent User Preferences for Human-Aligned Decision Making [4.1789291746171715]
  Human-aligned decision making requires accounting for both explicitly stated goals and latent user preferences.<n>We introduce CLIPR (Conversational Learning for Inferring Preferences and Reasoning), a framework that learns actionable, transferable natural language rules.<n> Evaluations on three datasets and a user study show that CLIPR consistently outperforms existing methods in improving alignment and reducing inference costs.
  arXiv  Detail & Related papers  (2026-05-12T19:32:10Z)
• CLIPer: Tailoring Diverse User Preference via Classifier-Guided Inference-Time Personalization [28.672568353736335]
  We introduce textbfCLIPer(textbfClassifier-guided textbfInference-time textbfPersonalization), a lightweight personalization approach.<n>Our method eliminates the need for extensive fine-tuning, inducing negligible additional computational overhead.
  arXiv  Detail & Related papers  (2026-05-08T02:47:30Z)
• Sparse Personalized Text Generation with Multi-Trajectory Reasoning [95.19317487799292]
  PAT (Personalization with Aligned Trajectories) is a reasoning framework for cold-start LLM personalization.<n>It first retrieves information along two complementary trajectories: writing-style cues from stylistically similar users and topic-specific context from preference-aligned users.<n>It then employs a reinforcement learning-based, iterative dual-reasoning mechanism that enables the LLM to jointly refine and integrate these signals.
  arXiv  Detail & Related papers  (2026-04-27T20:58:49Z)
• Instant Personalized Large Language Model Adaptation via Hypernetwork [56.512539596908745]
  Profile-to-PEFT is a scalable framework that employs a hypernetwork, trained end-to-end to map a user's encoded profile directly to a full set of adapter parameters.<n>We show that our method outperforms both prompt-based personalization and OPPU while using substantially fewer computational resources at deployment.
  arXiv  Detail & Related papers  (2025-10-18T00:41:25Z)
• CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steering [80.54309860395763]
  CoSteer is a novel collaborative framework that enables decoding-time personalization through localized delta steering.<n>We formulate token-level optimization as an online learning problem, where local delta vectors dynamically adjust the remote LLM's logits.<n>This approach preserves privacy by transmitting only the final steered tokens rather than raw data or intermediate vectors.
  arXiv  Detail & Related papers  (2025-07-07T08:32:29Z)
• Few-shot Steerable Alignment: Adapting Rewards and LLM Policies with Neural Processes [50.544186914115045]
  Large language models (LLMs) are increasingly embedded in everyday applications.<n> Ensuring their alignment with the diverse preferences of individual users has become a critical challenge.<n>We present a novel framework for few-shot steerable alignment.
  arXiv  Detail & Related papers  (2024-12-18T16:14:59Z)
• Context-aware Prompt Tuning: Advancing In-Context Learning with Adversarial Methods [69.36397993451742]
  This work introduces Context-aware Prompt Tuning (CPT), a method inspired by ICL, PT, and adversarial attacks. We modify specific context tokens, considering the unique structure of input and output formats. Inspired by adversarial attacks, we adjust the input based on the labels present in the context, focusing on minimizing, rather than maximizing, the loss.
  arXiv  Detail & Related papers  (2024-10-22T17:45:47Z)
• Personalized Adaptation via In-Context Preference Learning [20.042909385219716]
  Preference Pretrained Transformer (PPT) is a novel approach for adaptive personalization using online user feedback. Our results suggest the potential of in-context learning for scalable and efficient personalization in large language models.
  arXiv  Detail & Related papers  (2024-10-17T20:06:02Z)
• Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback [70.32795295142648]
  Linear alignment is a novel algorithm that aligns language models with human preferences in one single inference step. Experiments on both general and personalized preference datasets demonstrate that linear alignment significantly enhances the performance and efficiency of LLM alignment.
  arXiv  Detail & Related papers  (2024-01-21T10:46:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.