Locating and Controlling Implicit Personalization in Large Language Models

• URL: http://arxiv.org/abs/2608.11735v1
• Date: Wed, 12 Aug 2026 07:18:31 GMT
• Title: Locating and Controlling Implicit Personalization in Large Language Models
• Abstract summary: We show that a localized internal activation signal tracks changes in recommendations, with correlations up to r=0.87.<n>We show that removing the internal signal associated with one cue can suppress its influence, often more effectively than asking the model to ignore demographics via prompting.<n>These results connect implicit personalization behavior to an internal signal that can be analyzed and causally controlled.
• Score: 13.786400660073065
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) often shift their outputs in response to implicit demographic cues even when users never state a demographic identity. Previous work has documented this behavior, but the connection between these behavioral changes and the model's internal activations remains unclear. Using matched cued and neutral conversations across five LLMs, we establish that a localized internal activation signal tracks changes in recommendations, with correlations up to r=0.87. When multiple cues appear together, their internal signals largely combine, but the changes in output do not simply add up. We further show that removing the internal signal associated with one cue can suppress its influence, often more effectively than asking the model to ignore demographics via prompting, while largely preserving general benchmark performance. However, the ability to selectively remove one dimension's influence while leaving co-present dimensions intact remains highly model- and attribute-specific. These results connect implicit personalization behavior to an internal signal that can be analyzed and causally controlled.

Related papers

• A Low-Rank Subspace Analysis of LLM Interventions [21.607167713433622]
  Interventions designed to modify a particular behavior in LLMs often produce unintended changes in other behaviors.<n>This lack of targeted control makes it difficult to design and implement reliable safety controls.<n>We model behaviors as low-rank subspaces in activation space, and study how interventions influence across behaviors.
  arXiv  Detail & Related papers  (2026-06-12T12:24:25Z)
• Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering [87.43777061308658]
  We analyze predictive behavior in full spoken language models (FDSLMs)<n>We show that FDSLMs dynamically modulate their internal focus between two states: a generative state aligned with model output generation and a perceptive state aligned incoming user input.<n>During user interruptions, the model remains transiently biased toward the generative state before transitioning into the perceptive state, causing it miss the beginning of incoming input.
  arXiv  Detail & Related papers  (2026-06-09T19:08:07Z)
• Consistency Training while Mitigating Obfuscation via Rate Matching [6.354949885860511]
  Large language models are often influenced by extraneous input features, such as cues revealing a user's preferred answer.<n>Existing methods train for consistency over entire responses or internal activations, which also constrains whether the model verbalises said extraneous features.<n>We show this leads to obfuscation, where the model learns not to mention a cue while remaining influenced by it, which may undermine monitorability.<n>We introduce Rate Matching Consistency Training (RMCT), which trains for consistency over selected behavioural properties without constraining how this behaviour is expressed.
  arXiv  Detail & Related papers  (2026-06-01T13:10:49Z)
• Fair outputs, Biased Internals: Causal Potency and Asymmetry of Latent Bias in LLMs for High-Stakes Decisions [0.0]
  We investigate the use of open-weight models for mortgage underwriting using matched applications that differ only in racially-associated names.<n>We show that models show no output-level bias, yet retain and amplify demographic representations across model layers.<n>Through activation steering and novel cross-layer interventions, we demonstrate that this suppressed information is decision-relevant.
  arXiv  Detail & Related papers  (2026-05-12T12:14:58Z)
• From Packets to Patterns: Interpreting Encrypted Network Traffic as Longitudinal Behavioral Signals [2.8540604709894972]
  We test whether encrypted smartphone network traffic can passively capture behavioral patterns related to sleep, stress, and loneliness.<n>We model shared behavioral structure using a transformer backbone with per-user adapters.<n>We apply a sparse autoencoder to extract behavioral features corresponding to distinct patterns of activity.
  arXiv  Detail & Related papers  (2026-05-02T21:40:07Z)
• Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics [81.80010043113445]
  Local weight fine-tuning, LoRA-based adaptation, and activation-based interventions are studied in isolation.<n>We present a unified view that frames these interventions as dynamic weight updates induced by a control signal.<n>Across methods, we observe a consistent trade-off between preference and utility: stronger control increases preference while predictably reducing utility.
  arXiv  Detail & Related papers  (2026-02-02T17:04:36Z)
• User Hesitation and Negative Transfer in Multi-Behavior Recommendation [55.78729938627577]
  We propose a recommendation framework focused on weak signal learning, termed HNT.<n>By learning the characteristics of auxiliary behaviors that lead to target behaviors, HNT identifies similar auxiliary behaviors that did not trigger the target behavior.<n> Experiments on three real-world datasets demonstrate that HNT improves HR@10 and NDCG@10 by 12.57% and 14.37%, respectively.
  arXiv  Detail & Related papers  (2025-11-08T02:45:32Z)
• CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction [42.92011330807996]
  $textitCTR-Sink$ is a novel framework introducing behavior-level attention sinks tailored for recommendation scenarios.<n>Inspired by attention sink theory, it constructs attention focus sinks and dynamically regulates attention aggregation via external information.
  arXiv  Detail & Related papers  (2025-08-05T17:30:34Z)
• GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs [56.93583799109029]
  GrAInS is an inference-time steering approach that operates across both language-only and vision-language models and tasks.<n>During inference, GrAInS hidden activations at transformer layers guided by token-level attribution signals, and normalizes activations to preserve representational scale.<n>It consistently outperforms both fine-tuning and existing steering baselines.
  arXiv  Detail & Related papers  (2025-07-24T02:34:13Z)
• REAL: Reading Out Transformer Activations for Precise Localization in Language Model Steering [26.428347164111926]
  Inference-time steering aims to alter a large language model's responses without changing its parameters.<n>Existing approaches often rely on simplistic cues or ad hoc generalizations.<n>We introduce REAL, a framework for identifying behavior-relevant modules in Transformer models.
  arXiv  Detail & Related papers  (2025-06-10T02:16:50Z)
• Factual Self-Awareness in Language Models: Representation, Robustness, and Scaling [56.26834106704781]
  Factual incorrectness in generated content is one of the primary concerns in ubiquitous deployment of large language models (LLMs)<n>We provide evidence supporting the presence of LLMs' internal compass that dictate the correctness of factual recall at the time of generation.<n>Scaling experiments across model sizes and training dynamics highlight that self-awareness emerges rapidly during training and peaks in intermediate layers.
  arXiv  Detail & Related papers  (2025-05-27T16:24:02Z)
• Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles [104.60508550106618]
  We propose DiffDiv, an ensemble diversification framework exploiting Diffusion Probabilistic Models (DPMs)<n>We show that DPMs can generate images with novel feature combinations, even when trained on samples displaying correlated input features.<n>We show that DPM-guided diversification is sufficient to remove dependence on shortcut cues, without a need for additional supervised signals.
  arXiv  Detail & Related papers  (2023-11-23T15:47:33Z)
• Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation [102.24108167002252]
  We propose a novel attention network, named self-modulating attention, that models the complex and non-linearly evolving dynamic user preferences. We empirically demonstrate the effectiveness of our method on top-N sequential recommendation tasks, and the results on three large-scale real-world datasets show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-30T03:54:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.