Multi-ResNets for Subspace Preconditioning in Constrained Optimization

• URL: http://arxiv.org/abs/2606.06300v1
• Date: Thu, 04 Jun 2026 15:37:55 GMT
• Title: Multi-ResNets for Subspace Preconditioning in Constrained Optimization
• Abstract summary: MResOpt is a staged residual neural network architecture for constrained optimization problems.<n>We show that MResOpt supports a learned division labor that keeps iterates on the equality constraint manifold.
• Score: 3.874563682396874
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose MResOpt, a staged residual neural network architecture for constrained optimization problems. Our architecture fits within predict-complete-correct pipelines and decomposes constraint satisfaction by priority through intermediate re-completion and stage-aware losses. The framework enables domain-informed ordered constraint satisfaction which allows the network to utilize ordinal structure when present. Under an idealized infinite-width regime, we show that our design behaves as sequential Gaussian Process regression. On synthetic QP, QCQP, and SOCP benchmarks, the staged architecture improves high-priority constraint satisfaction across convex and non-convex settings. On line-flow-constrained AC optimal power flow, we introduce a physics-motivated constraint ordering and show that MResOpt supports a learned division of labor that keeps iterates on the equality manifold, achieving substantially lower high-priority violation than reprojected baselines while remaining computationally efficient.

Related papers

• Optimality-Preserving Decomposition for Scalable QAOA in Natural-Language-Guided Multi-Drone Assignment [1.189955933770711]
  We present an end-to-end framework integrating a fine-tuned Large Language Model front-end with a highly scalable, domain-specific quantum-classical backend.<n>Our framework features a novel constraint-preserving graph partitioner and a compressed separator-based dynamic programming (DP) merge.<n> Empirical results demonstrate that this architecture circumvents classical scaling walls, recovering the global optimum on 100% of idealized cases and 96.3% under real QAOA sampling.
  arXiv  Detail & Related papers  (2026-06-12T08:31:14Z)
• Quantum Tilted Loss in Variational Optimization: Theory and Applications [2.274915755738124]
  We introduce the Quantum Tilted Loss (QTL) to reshape the optimization landscape.<n>QTL can amplify gradient signals in structured settings while preserving the problem's true global minima.<n>We formalize this trainability-estimability trade-off, demonstrating how aggressive tilting fundamentally shifts the optimization bottleneck.
  arXiv  Detail & Related papers  (2026-05-04T17:25:43Z)
• Optimizing Entanglement Distribution Protocols: Maximizing Classical Information in Quantum Networks [5.621690515150937]
  Efficient entanglement distribution is the foundational challenge in realizing large-scale Quantum Networks.<n>This paper addresses the entanglement distribution problem by introducing four pivotal advances.<n>First, recognizing that the primary application of quantum communication is the transmission of private information, we derive the Ensemble Capacity (EC), a novel metric that explicitly quantifies the secure classical information enabled by the entanglement distribution.
  arXiv  Detail & Related papers  (2026-03-26T12:08:13Z)
• SpanNorm: Reconciling Training Stability and Performance in Deep Transformers [55.100133502295996]
  We propose SpanNorm, a novel technique designed to resolve the dilemma by integrating the strengths of both paradigms.<n>We provide a theoretical analysis demonstrating that SpanNorm, combined with a principled scaling strategy, maintains bounded signal variance throughout the network.<n> Empirically, SpanNorm consistently outperforms standard normalization schemes in both dense and Mixture-of-Experts (MoE) scenarios.
  arXiv  Detail & Related papers  (2026-01-30T05:21:57Z)
• Robust Stochastically-Descending Unrolled Networks [85.6993263983062]
  Deep unrolling is an emerging learning-to-optimize method that unrolls a truncated iterative algorithm in the layers of a trainable neural network.<n>We show that convergence guarantees and generalizability of the unrolled networks are still open theoretical problems.<n>We numerically assess unrolled architectures trained under the proposed constraints in two different applications.
  arXiv  Detail & Related papers  (2023-12-25T18:51:23Z)
• Neural Quantile Optimization for Edge-Cloud Networking [13.509945075582447]
  We seek the best traffic allocation scheme for the edge-cloud computing network that satisfies constraints and minimizes the cost based on burstable billing. We introduce the Gumbel-softmax sampling network to solve the optimization problems via unsupervised learning. The trained network works as an efficient traffic allocation scheme sampler, remarkably outperforming the random strategy in feasibility and cost function value.
  arXiv  Detail & Related papers  (2023-07-11T11:05:10Z)
• Optimal Sets and Solution Paths of ReLU Networks [56.40911684005949]
  We develop an analytical framework to characterize the set of optimal ReLU networks. We establish conditions for the neuralization of ReLU networks to be continuous, and develop sensitivity results for ReLU networks.
  arXiv  Detail & Related papers  (2023-05-31T18:48:16Z)
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
  Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
  arXiv  Detail & Related papers  (2023-03-16T21:06:13Z)
• Robust Deep Compressive Sensing with Recurrent-Residual Structural Constraints [0.0]
  Existing deep sensing (CS) methods either ignore adaptive online optimization or depend on costly iterative reconstruction. This work explores a novel image CS framework with recurrent-residual structural constraint, termed as R$2$CS-NET. As the first deep CS framework efficiently bridging adaptive online optimization, the R$2$CS-NET integrates the robustness of online optimization with the efficiency and nonlinear capacity of deep learning methods.
  arXiv  Detail & Related papers  (2022-07-15T05:56:13Z)
• Joint inference and input optimization in equilibrium networks [68.63726855991052]
  deep equilibrium model is a class of models that foregoes traditional network depth and instead computes the output of a network by finding the fixed point of a single nonlinear layer. We show that there is a natural synergy between these two settings. We demonstrate this strategy on various tasks such as training generative models while optimizing over latent codes, training models for inverse problems like denoising and inpainting, adversarial training and gradient based meta-learning.
  arXiv  Detail & Related papers  (2021-11-25T19:59:33Z)
• Efficient Micro-Structured Weight Unification and Pruning for Neural Network Compression [56.83861738731913]
  Deep Neural Network (DNN) models are essential for practical applications, especially for resource limited devices. Previous unstructured or structured weight pruning methods can hardly truly accelerate inference. We propose a generalized weight unification framework at a hardware compatible micro-structured level to achieve high amount of compression and acceleration.
  arXiv  Detail & Related papers  (2021-06-15T17:22:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.