Soft-Radial Projection for Constrained End-to-End Learning

• URL: http://arxiv.org/abs/2602.03461v1
• Date: Tue, 03 Feb 2026 12:33:44 GMT
• Title: Soft-Radial Projection for Constrained End-to-End Learning
• Authors: Philipp J. Schneider, Daniel Kuhn,
• Abstract summary: We introduce Soft-Radial Projection, a differentiable re parameterization layer that circumvents gradient saturation.<n>This construction guarantees strict feasibility while preserving a full-rank Jacobian almost everywhere.<n>We empirically show improved convergence behavior and solution quality over state-of-the-art optimization- and projection-based baselines.
• Score: 2.3367876359631645
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Integrating hard constraints into deep learning is essential for safety-critical systems. Yet existing constructive layers that project predictions onto constraint boundaries face a fundamental bottleneck: gradient saturation. By collapsing exterior points onto lower-dimensional surfaces, standard orthogonal projections induce rank-deficient Jacobians, which nullify gradients orthogonal to active constraints and hinder optimization. We introduce Soft-Radial Projection, a differentiable reparameterization layer that circumvents this issue through a radial mapping from Euclidean space into the interior of the feasible set. This construction guarantees strict feasibility while preserving a full-rank Jacobian almost everywhere, thereby preventing the optimization stalls typical of boundary-based methods. We theoretically prove that the architecture retains the universal approximation property and empirically show improved convergence behavior and solution quality over state-of-the-art optimization- and projection-based baselines.

Related papers

• Global Optimization By Gradient from Hierarchical Score-Matching Spaces [6.444818462799464]
  This work solves all optimization problems with various complex constraints as a general hierarchical optimization objective without constraints.<n>It reveals the profound connection between global optimization and diffusion based generative modeling.
  arXiv  Detail & Related papers  (2026-01-14T15:26:01Z)
• Data-Driven Adaptive Gradient Recovery for Unstructured Finite Volume Computations [0.0]
  We present a novel data-driven approach for enhancing gradient reconstruction in unstructured finite volume methods for hyperbolic conservation laws.<n>Our approach extends previous structured-grid methodologies to unstructured meshes through a modified DeepONet architecture.<n>The proposed algorithm is faster and more accurate than the traditional second-order finite volume solver.
  arXiv  Detail & Related papers  (2025-07-22T13:23:57Z)
• HiNeuS: High-fidelity Neural Surface Mitigating Low-texture and Reflective Ambiguity [8.74691272469226]
  HiNeuS is a unified framework that holistically addresses three core limitations in existing approaches.<n>We introduce: 1) Differential visibility verification through SDF-guided ray tracing; 2) Planar-conformal regularization via ray-aligned geometry patches; and 3) Physically-grounded Eikonal relaxation that dynamically modulates geometric constraints based on local gradients.
  arXiv  Detail & Related papers  (2025-06-30T13:45:25Z)
• Mirror Descent Under Generalized Smoothness [23.5387392871236]
  We introduce a new $ell*$-smoothness concept that measures the norm of Hessians in terms of a general norm and its dual.<n>We establish convergence for mirror-descent-type algorithms, matching the rates under the classic smoothness.
  arXiv  Detail & Related papers  (2025-02-02T11:23:10Z)
• Generalization Bounds of Surrogate Policies for Combinatorial Optimization Problems [53.03951222945921]
  We analyze smoothed (perturbed) policies, adding controlled random perturbations to the direction used by the linear oracle.<n>Our main contribution is a generalization bound that decomposes the excess risk into perturbation bias, statistical estimation error, and optimization error.<n>We illustrate the scope of the results on applications such as vehicle scheduling, highlighting how smoothing enables both tractable training and controlled generalization.
  arXiv  Detail & Related papers  (2024-07-24T12:00:30Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Projection-Free Online Convex Optimization with Stochastic Constraints [0.0]
  We develop projection-free algorithms for online convex optimization with constraints. We deduce sublinear regret and constraint violation bounds for various settings. We prove that the constraint violation can be reduced to have the same growth as the regret.
  arXiv  Detail & Related papers  (2023-05-02T11:27:34Z)
• Multilevel Geometric Optimization for Regularised Constrained Linear Inverse Problems [3.3161769688599025]
  We consider a hierarchy of models with varying discretization levels. Finer models are accurate but expensive to compute, while coarser models are less accurate but cheaper to compute. When working at the fine level, multilevel optimisation computes the search direction based on a coarser model which speeds up updates at the fine level.
  arXiv  Detail & Related papers  (2022-07-11T15:15:33Z)
• Inverse design of photonic devices with strict foundry fabrication constraints [55.41644538483948]
  We introduce a new method for inverse design of nanophotonic devices which guarantees that designs satisfy strict length scale constraints. We demonstrate the performance and reliability of our method by designing several common integrated photonic components.
  arXiv  Detail & Related papers  (2022-01-31T02:27:25Z)
• On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systems [99.59934203759754]
  We introduce a class of first-order methods for smooth constrained optimization. Two distinctive features of our approach are that projections or optimizations over the entire feasible set are avoided. The resulting algorithmic procedure is simple to implement even when constraints are nonlinear.
  arXiv  Detail & Related papers  (2021-07-17T11:45:13Z)
• Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization [98.0595480384208]
  We propose a generalization extraient spaces which converges to a stationary point. The algorithm applies not only to general $p$-normed spaces, but also to general $p$-dimensional vector spaces.
  arXiv  Detail & Related papers  (2020-10-31T21:35:42Z)
• Cogradient Descent for Bilinear Optimization [124.45816011848096]
  We introduce a Cogradient Descent algorithm (CoGD) to address the bilinear problem. We solve one variable by considering its coupling relationship with the other, leading to a synchronous gradient descent. Our algorithm is applied to solve problems with one variable under the sparsity constraint.
  arXiv  Detail & Related papers  (2020-06-16T13:41:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.