Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization

• URL: http://arxiv.org/abs/2604.04231v1
• Date: Sun, 05 Apr 2026 19:27:24 GMT
• Title: Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization
• Abstract summary: We propose a subspace control framework for constrained model training.<n>SIFT (spectral interference-free training) consistently achieves substantial and robust performance improvements across all tasks.
• Score: 27.280175507074322
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models, such as large language models (LLMs), are powerful but often require customization before deployment to satisfy practical constraints such as safety, privacy, and task-specific requirements, leading to "constrained" optimization problems for model steering and adaptation. However, solving such problems remains largely underexplored and is particularly challenging due to interference between the primary objective and constraint objectives during optimization. In this paper, we propose a subspace control framework for constrained model training. Specifically, (i) we first analyze, from a model merging perspective, how spectral cross-task interference arises and show that it can be resolved via a one-shot solution that orthogonalizes the merged subspace; (ii) we establish a connection between this solution and gradient orthogonalization in the spectral optimizer Muon; and (iii) building on these insights, we introduce SIFT (spectral interference-free training), which leverages a localization scheme to selectively intervene during optimization, enabling controllable updates that mitigate objective-constraint conflicts. We evaluate SIFT across four representative applications: (a) machine unlearning, (b) safety alignment, (c) text-to-speech adaptation, and (d) hallucination mitigation. Compared to both control-based and control-free baselines, SIFT consistently achieves substantial and robust performance improvements across all tasks. Code is available at https://github.com/OPTML-Group/SIFT.

Related papers

• UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling [35.728353523103074]
  We introduce Unified Inference Scaling (UIS), which unifies model routing and test-time scaling.<n>We show that UniScale effectively exploits the synergy in the UIS space to deliver a fine-grained and consistently better quality-cost trade-off.
  arXiv  Detail & Related papers  (2026-05-29T06:31:21Z)
• Language Generation as Optimal Control: Closed-Loop Diffusion in Latent Control Space [99.30250506439678]
  This work reformulates language generation as an optimal control problem.<n>It provides a unified theoretical perspective to analyze autoregressive and diffusion models.
  arXiv  Detail & Related papers  (2026-05-14T08:13:43Z)
• Multi-Objective Bayesian Optimization via Adaptive \varepsilon-Constraints Decomposition [19.76344805798142]
  We propose textitSTAGE-BO, Sequential Targeting Adaptive Gap-Filling $varepsilon$-Constraint Bayesian Optimization.<n>By analyzing the coverage of the approximate Pareto front, our method identifies the largest geometric gaps.<n>These gaps are then used as constraints, which transforms the problem into a sequence of inequality-constrained subproblems.
  arXiv  Detail & Related papers  (2026-04-17T11:24:56Z)
• Feasibility Restoration under Conflicting STL Specifications with Pareto-Optimal Refinement [18.383508411056944]
  Signal Temporal Logic (STL) is expressive language that specifies expressive requirements in robotics.<n>STL specifications may become conflicting in real-world applications, where safety rules, traffic regulations, task and objectives can be cannot be satisfied together.<n>We propose a unified two-stage framework that first restores feasibility via minimal relaxation, then refine the feasible solution by formulating it as a value-aware multi-objective optimization problem.
  arXiv  Detail & Related papers  (2026-03-06T23:41:01Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• Adaptive Linear Path Model-Based Diffusion [52.84663832658799]
  We introduce Linear Path Model-Based Diffusion (LP-MBD), which replaces the variance-preserving schedule with a flow-matching linear probability path.<n>We also propose Adaptive LP-MBD (ALP-MBD), which leverages reinforcement learning to adjust diffusion steps and noise levels according to task complexity and environmental conditions.
  arXiv  Detail & Related papers  (2026-02-02T21:33:03Z)
• Automated Optimization Modeling via a Localizable Error-Driven Perspective [20.591721861026414]
  We propose a novel error-driven learning framework for automated optimization modeling.<n>MIND customized the whole model training framework from data synthesis to post-training.<n>MIND consistently outperforms all the state-of-the-art automated optimization modeling approaches.
  arXiv  Detail & Related papers  (2026-01-17T09:59:01Z)
• QoS-Aware Hierarchical Reinforcement Learning for Joint Link Selection and Trajectory Optimization in SAGIN-Supported UAV Mobility Management [52.15690855486153]
  A space-air-ground integrated network (SAGIN) has emerged as an essential architecture for enabling ubiquitous UAV connectivity.<n>This paper formulates UAV mobility management in SAGIN as a constrained multiobjective joint optimization problem.
  arXiv  Detail & Related papers  (2025-12-17T06:22:46Z)
• Steerable Adversarial Scenario Generation through Test-Time Preference Alignment [58.37104890690234]
  Adversarial scenario generation is a cost-effective approach for safety assessment of autonomous driving systems.<n>We introduce a new framework named textbfSteerable textbfAdversarial scenario textbfGEnerator (SAGE)<n>SAGE enables fine-grained test-time control over the trade-off between adversariality and realism without any retraining.
  arXiv  Detail & Related papers  (2025-09-24T13:27:35Z)
• Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning [132.7040981721302]
  We study the Constrained Convex Decision Process (MDP), where the goal is to minimize a convex functional of the visitation measure. Design algorithms for a constrained convex MDP faces several challenges, including handling the large state space.
  arXiv  Detail & Related papers  (2024-02-16T16:35:18Z)
• Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment [46.44464839353993]
  We introduce Rewards-in-Context (RiC), which conditions the response of a foundation model on multiple rewards in its prompt context. RiC only requires supervised fine-tuning of a single foundation model and supports dynamic adjustment for user preferences during inference time.
  arXiv  Detail & Related papers  (2024-02-15T18:58:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.