Prism-$Δ$: Differential Subspace Steering for Prompt Highlighting in Large Language Models

• URL: http://arxiv.org/abs/2603.10705v1
• Date: Wed, 11 Mar 2026 12:24:45 GMT
• Title: Prism-$Δ$: Differential Subspace Steering for Prompt Highlighting in Large Language Models
• Abstract summary: We propose PRISM-$$ (Projection-based Relevance-Informed Steering Method) to maximize discriminative energy while eliminating shared directions.<n> PRISM-$$ matches or exceeds the best existing method on 19 of 20 configurations, with relative gains up to +10.6%, while halving the cost of steering.<n> PRISM-$$ is compatible with FlashAttention and adds negligible memory overhead.
• Score: 85.18197548789291
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompt highlighting steers a large language model to prioritize user-specified text spans during generation. A key challenge is extracting steering directions that capture the difference between relevant and irrelevant contexts, rather than shared structural patterns common to both. We propose PRISM-$Δ$ (Projection-based Relevance-Informed Steering Method), which decomposes the difference between positive and negative cross-covariance matrices to maximize discriminative energy while eliminating shared directions. Each attention head receives a continuous softplus importance weight, letting weak-but-useful heads contribute at reduced strength. The framework extends naturally to Value representations, capturing content-channel signal that Key-only methods leave unused. Across four benchmarks and five models, PRISM-$Δ$ matches or exceeds the best existing method on 19 of 20 configurations, with relative gains up to +10.6%, while halving the fluency cost of steering. PRISM-$Δ$ also scales to long-context retrieval, outperforming the best existing method by up to +4.8% relative gain. PRISM-$Δ$ is compatible with FlashAttention and adds negligible memory overhead.

Related papers

• Beyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse Autoencoders [9.291632832739822]
  We introduce two sparsity regularizers compatible with the Top-$k$ architecture.<n>Our central finding is that hard architectural sparsity and soft sparsity regularization are complementary rather than mutually exclusive.
  arXiv  Detail & Related papers  (2026-06-25T17:34:39Z)
• Token-Efficient Multimodal Reasoning via Image Prompt Packaging [0.6465251961564605]
  We introduce Image Prompt Packaging, a prompting paradigm that embeds structured text directly into images to reduce text token overhead.<n>We benchmark it across five datasets, three frontier models, and two task families.<n>We derive a cost decomposing savings by token type and show IPPg achieves 35.8--91.0% inference cost reductions.
  arXiv  Detail & Related papers  (2026-04-02T19:50:59Z)
• Multiple-play Stochastic Bandits with Prioritized Arm Capacity Sharing [52.124267908936396]
  The model is composed of $M$ arms and $K$ plays.<n>Each arm has a number of capacities, and each unit of capacity is associated with a reward function.<n>When multiple plays compete for the arm capacity, the arm capacity is allocated in a larger priority weight first manner.
  arXiv  Detail & Related papers  (2025-12-25T11:19:09Z)
• Repulsor: Accelerating Generative Modeling with a Contrastive Memory Bank [65.00301565190824]
  mname is a plug-and-play training framework that requires no external encoders.<n>mname achieves a state-of-the-art FID of textbf2.40 within 400k steps, significantly outperforming comparable methods.
  arXiv  Detail & Related papers  (2025-12-09T14:39:26Z)
• Preference-based Reinforcement Learning beyond Pairwise Comparisons: Benefits of Multiple Options [35.41703011973504]
  We study online preference-based reinforcement learning (PbRL) with the goal of improving sample efficiency.<n>We propose M-AUPO, an algorithm that selects multiple actions by maximizing the average uncertainty within the offered subset.<n>We prove that M-AUPO achieves a suboptimality gap of $tildemathcalOleft( fracdT sqrt sum_t=1T frac1|S_t| right)$, where $T$ is the total number of rounds, $d$
  arXiv  Detail & Related papers  (2025-10-21T15:11:01Z)
• ResT: Reshaping Token-Level Policy Gradients for Tool-Use Large Language Models [62.82372407840088]
  Large language models (LLMs) transcend passive generation and act as goal-directed agents by invoking external tools.<n>textbfReshaped textbfToken-level policy gradients (textbfResT) for tool-use tasks.<n>textbfResT achieves state-of-the-art results, outperforming prior methods by up to $8.76%$.
  arXiv  Detail & Related papers  (2025-09-26T03:38:27Z)
• FLARE: Robot Learning with Implicit World Modeling [87.81846091038676]
  $textbfFLARE$ integrates predictive latent world modeling into robot policy learning.<n>$textbfFLARE$ achieves state-of-the-art performance, outperforming prior policy learning baselines by up to 26%.<n>Our results establish $textbfFLARE$ as a general and scalable approach for combining implicit world modeling with high-frequency robotic control.
  arXiv  Detail & Related papers  (2025-05-21T15:33:27Z)
• InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion [36.27704594180795]
  InfiGFusion is a structure-aware fusion framework with a novel textitGraph-on-Logits Distillation (GLD) loss.<n>We show that GLD consistently improves fusion quality and stability.<n>It shows particular strength in complex reasoning tasks, with +35.6 improvement on Multistep Arithmetic and +37.06 on Causal Judgement over SFT.
  arXiv  Detail & Related papers  (2025-05-20T03:55:35Z)
• The Differences Between Direct Alignment Algorithms are a Blur [3.0059120458540383]
  We show that one-stage methods (e.g. ORPO, ASFT) underperform compared to two-stage approaches.<n>We demonstrate that adapting them to a two-stage setup with an explicit SFT phase can improve their performance.<n>Our comprehensive analysis reveals that the choice between pairwise and pointwise objectives is the primary determinant of alignment success.
  arXiv  Detail & Related papers  (2025-02-03T10:54:14Z)
• Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions [26.025283259518936]
  Rodimus is a new type of attention-based large language models (LLMs)<n>Rodimus employs a data-dependent tempered selection mechanism within a linear attention-based, purely recurrent framework.<n>Rodimus$+$ combines Rodimus with the innovative Sliding Window Shared-Key Attention (SW-SKA) in a hybrid approach.<n>Our experiments demonstrate that Rodimus$+$-1.6B, trained on 1 trillion tokens, achieves superior downstream performance against models trained on more tokens.
  arXiv  Detail & Related papers  (2024-10-09T06:22:36Z)
• Inference-Time Language Model Alignment via Integrated Value Guidance [21.294207620111525]
  We introduce $textitIntegrated Value Guidance$ (IVG) IVG uses implicit and explicit value functions to guide language model decoding at token and chunk-level. We demonstrate IVG's versatility across various tasks.
  arXiv  Detail & Related papers  (2024-09-26T13:15:18Z)
• Nearly Optimal Algorithms for Contextual Dueling Bandits from Adversarial Feedback [58.66941279460248]
  Learning from human feedback plays an important role in aligning generative models, such as large language models (LLM)<n>We study a model within this domain--contextual dueling bandits with adversarial feedback, where the true preference label can be flipped by an adversary.<n>We propose an algorithm namely robust contextual dueling bandits (RCDB), which is based on uncertainty-weighted maximum likelihood estimation.
  arXiv  Detail & Related papers  (2024-04-16T17:59:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.