PRISM: Structured Optimization via Anisotropic Spectral Shaping

• URL: http://arxiv.org/abs/2602.03096v1
• Date: Tue, 03 Feb 2026 04:41:11 GMT
• Title: PRISM: Structured Optimization via Anisotropic Spectral Shaping
• Authors: Yujie Yang,
• Abstract summary: PRISM is an efficient, low-rank-second-order preconditioner.<n>It adaptively suppresses updates in high-variance subspaces while preserving update strength in signal-dominated directions.
• Score: 10.078746583283754
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose PRISM, an optimizer that enhances first-order spectral descent methods like Muon with partial second-order information. It constructs an efficient, low-rank quasi-second-order preconditioner via innovation-augmented polar decomposition. This mechanism enables PRISM to perform anisotropic spectral shaping, which adaptively suppresses updates in high-variance subspaces while preserving update strength in signal-dominated directions. Crucially, this is achieved with minimal computational overhead and zero additional memory compared to first-order baselines. PRISM demonstrates a practical strategy for integrating curvature-adaptive properties into the spectral optimization paradigm.

Related papers

• Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization [40.95701844244596]
  We show that ZO optimization can be substantially improved by unifying two complementary principles.<n>We instantiate in a new method, ZO-Muon, admitting a natural interpretation as a low-rank Muon in the ZO setting.
  arXiv  Detail & Related papers  (2026-02-19T08:08:33Z)
• Muon with Spectral Guidance: Efficient Optimization for Scientific Machine Learning [10.647088281181222]
  SpecMuon is a spectral-aware, multi-mode gradient flow for physics-informed learning.<n>It regulates step sizes according to the global loss energy while preserving Muon's scale-balancing properties.<n>It achieves faster convergence and improved stability compared with Adam AdamW.
  arXiv  Detail & Related papers  (2026-02-18T03:56:20Z)
• Towards a Unified Analysis of Neural Networks in Nonparametric Instrumental Variable Regression: Optimization and Generalization [66.08522228989634]
  We establish the first global convergence result of neural networks for two stage least squares (2SLS) approach in nonparametric instrumental variable regression (NPIV)<n>This is achieved by adopting a lifted perspective through mean-field Langevin dynamics (MFLD)
  arXiv  Detail & Related papers  (2025-11-18T17:51:17Z)
• Natural Spectral Fusion: p-Exponent Cyclic Scheduling and Early Decision-Boundary Alignment in First-Order Optimization [11.323131201168572]
  We propose Natural Spectral Fusion (NSF): reframing training as controllable spectral coverage and information fusion.<n>NSF has two core principles: treating the balances as a spectral controller that dynamically low- and high-frequency information.<n>We show that cyclic scheduling consistently reduces test error and demonstrates distinct convergence behavior.
  arXiv  Detail & Related papers  (2025-09-05T00:00:00Z)
• Energy Landscape Plummeting in Variational Quantum Eigensolver: Subspace Optimization, Non-iterative Corrections and Generator-informed Initialization for Improved Quantum Efficiency [0.0]
  Variational Quantum Eigensolver (VQE) faces significant challenges due to hardware noise and the presence of barren plateaus and local traps.<n>We introduce a general formalism that optimize hardware resource utilization and accuracy by projecting VQE optimizations on to a reduced-dimensional subspace.<n> Numerical simulations show that, when integrated with any chemistry-inspired ansatz, our method can provide one to two orders of magnitude better estimation of the minima.
  arXiv  Detail & Related papers  (2025-04-17T17:07:09Z)
• Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization [78.82586283794886]
  $chi2$-Preference Optimization ($chi$PO) is an efficient offline alignment algorithm provably robust to overoptimization.<n>$chi$PO implements the principle of pessimism in the face of uncertainty via regularization.<n>$chi$PO's simplicity and strong guarantees make it the first practical and general-purpose offline alignment algorithm provably robust to overoptimization.
  arXiv  Detail & Related papers  (2024-07-18T11:08:40Z)
• Memory-Efficient Optimization with Factorized Hamiltonian Descent [11.01832755213396]
  We introduce a novel adaptive, H-Fac, which incorporates a memory-efficient factorization approach to address this challenge. By employing a rank-1 parameterization for both momentum and scaling parameter estimators, H-Fac reduces memory costs to a sublinear level. We develop our algorithms based on principles derived from Hamiltonian dynamics, providing robust theoretical underpinnings in optimization dynamics and convergence guarantees.
  arXiv  Detail & Related papers  (2024-06-14T12:05:17Z)
• Generalized Preference Optimization: A Unified Approach to Offline Alignment [54.97015778517253]
  We propose generalized preference optimization (GPO), a family of offline losses parameterized by a general class of convex functions. GPO enables a unified view over preference optimization, encompassing existing algorithms such as DPO, IPO and SLiC as special cases. Our results present new algorithmic toolkits and empirical insights to alignment practitioners.
  arXiv  Detail & Related papers  (2024-02-08T15:33:09Z)
• Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers [109.52244418498974]
  We propose a novel textscAdmeta (textbfADouble exponential textbfMov averagtextbfE textbfAdaptive and non-adaptive momentum) framework. We provide two implementations, textscAdmetaR and textscAdmetaS, the former based on RAdam and the latter based on SGDM.
  arXiv  Detail & Related papers  (2023-07-02T18:16:06Z)
• Nesterov Meets Optimism: Rate-Optimal Separable Minimax Optimization [108.35402316802765]
  We propose a new first-order optimization algorithm -- AcceleratedGradient-OptimisticGradient (AG-OG) Ascent. We show that AG-OG achieves the optimal convergence rate (up to a constant) for a variety of settings. We extend our algorithm to extend the setting and achieve the optimal convergence rate in both bi-SC-SC and bi-C-SC settings.
  arXiv  Detail & Related papers  (2022-10-31T17:59:29Z)
• Amortized Proximal Optimization [11.441395750267052]
  Amortized Proximal Optimization (APO) is a framework for online meta-optimization of parameters that govern optimization. We show how APO can be used to adapt a learning rate or a structured preconditioning matrix. We empirically test APO for online adaptation of learning rates and structured preconditioning for regression, image reconstruction, image classification, and natural language translation tasks.
  arXiv  Detail & Related papers  (2022-02-28T20:50:48Z)
• Optimistic Policy Optimization is Provably Efficient in Non-stationary MDPs [113.8752163061151]
  We study episodic reinforcement learning (RL) in non-stationary linear kernel Markov decision processes (MDPs)<n>We propose the underlineperiodically underlinerestarted underlineoptimistic underlinepolicy underlineoptimization algorithm (PROPO)<n>PROPO features two mechanisms: sliding-window-based policy evaluation and periodic-restart-based policy improvement.
  arXiv  Detail & Related papers  (2021-10-18T02:33:20Z)
• EBM-Fold: Fully-Differentiable Protein Folding Powered by Energy-based Models [53.17320541056843]
  We propose a fully-differentiable approach for protein structure optimization, guided by a data-driven generative network. Our EBM-Fold approach can efficiently produce high-quality decoys, compared against traditional Rosetta-based structure optimization routines.
  arXiv  Detail & Related papers  (2021-05-11T03:40:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.