ENFORCE: Nonlinear Constrained Learning with Adaptive-depth Neural Projection

• URL: http://arxiv.org/abs/2502.06774v3
• Date: Thu, 15 May 2025 20:21:20 GMT
• Title: ENFORCE: Nonlinear Constrained Learning with Adaptive-depth Neural Projection
• Authors: Giacomo Lastrucci, Artur M. Schweidtmann,
• Abstract summary: We introduce ENFORCE, a neural network architecture that uses an adaptive projection module (AdaNP) to enforce nonlinear equality constraints in the predictions.<n>We prove that our projection mapping is 1-Lipschitz, making it well-suited for stable training.<n>The predictions of our new architecture satisfy $N_C$ equality constraints that are nonlinear in both the inputs and outputs of the neural network.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensuring neural networks adhere to domain-specific constraints is crucial for addressing safety and ethical concerns while also enhancing inference accuracy. Despite the nonlinear nature of most real-world tasks, existing methods are predominantly limited to affine or convex constraints. We introduce ENFORCE, a neural network architecture that uses an adaptive projection module (AdaNP) to enforce nonlinear equality constraints in the predictions. We prove that our projection mapping is 1-Lipschitz, making it well-suited for stable training. We evaluate ENFORCE on an illustrative regression task and for learning solutions to high-dimensional optimization problems in an unsupervised setting. The predictions of our new architecture satisfy $N_C$ equality constraints that are nonlinear in both the inputs and outputs of the neural network, while maintaining scalability with a tractable computational complexity of $\mathcal{O}(N_C^3)$ at training and inference time.

Related papers

• From Initial Data to Boundary Layers: Neural Networks for Nonlinear Hyperbolic Conservation Laws [0.0]
  We address the approximation of entropy solutions to initial-boundary value problems for nonlinear strictly hyperbolic conservation laws using neural networks.<n>A general and systematic framework is introduced for the design of efficient and reliable learning algorithms, combining fast convergence during training with accurate predictions.
  arXiv  Detail & Related papers  (2025-06-02T09:12:13Z)
• Enforcing Hard Linear Constraints in Deep Learning Models with Decision Rules [8.098452803458253]
  This paper proposes a model-agnostic framework for enforcing input-dependent linear equality and inequality constraints on neural network outputs.<n>The architecture combines a task network trained for prediction accuracy with a safe network trained using decision rules from the runtime and robust optimization to ensure feasibility across the entire input space.
  arXiv  Detail & Related papers  (2025-05-20T03:09:44Z)
• Hard-Constrained Neural Networks with Universal Approximation Guarantees [5.3663546125491735]
  HardNet is a framework for constructing neural networks that inherently satisfy hard constraints without sacrificing model capacity. We show that HardNet retains the universal approximation capabilities of neural networks.
  arXiv  Detail & Related papers  (2024-10-14T17:59:24Z)
• LinSATNet: The Positive Linear Satisfiability Neural Networks [116.65291739666303]
  This paper studies how to introduce the popular positive linear satisfiability to neural networks. We propose the first differentiable satisfiability layer based on an extension of the classic Sinkhorn algorithm for jointly encoding multiple sets of marginal distributions.
  arXiv  Detail & Related papers  (2024-07-18T22:05:21Z)
• N-Adaptive Ritz Method: A Neural Network Enriched Partition of Unity for Boundary Value Problems [1.2200609701777907]
  This work introduces a novel neural network-enriched Partition of Unity (NN-PU) approach for solving boundary value problems via artificial neural networks. The NN enrichment is constructed by combining pre-trained feature-encoded NN blocks with an untrained NN block. The proposed method offers accurate solutions while notably reducing the computational cost compared to the conventional adaptive refinement in the mesh-based methods.
  arXiv  Detail & Related papers  (2024-01-16T18:11:14Z)
• The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models [75.33431791218302]
  Deep Neural Network Network (DNN) models are used for programming purposes. In this paper we examine the use of convex neural recovery models. We show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program. We also show that all the stationary non-dimensional objective objective can be characterized as the standard a global subsampled convex solvers program.
  arXiv  Detail & Related papers  (2023-12-19T23:04:56Z)
• Achieving Constraints in Neural Networks: A Stochastic Augmented Lagrangian Approach [49.1574468325115]
  Regularizing Deep Neural Networks (DNNs) is essential for improving generalizability and preventing overfitting. We propose a novel approach to DNN regularization by framing the training process as a constrained optimization problem. We employ the Augmented Lagrangian (SAL) method to achieve a more flexible and efficient regularization mechanism.
  arXiv  Detail & Related papers  (2023-10-25T13:55:35Z)
• The Boundaries of Verifiable Accuracy, Robustness, and Generalisation in Deep Learning [71.14237199051276]
  We consider classical distribution-agnostic framework and algorithms minimising empirical risks. We show that there is a large family of tasks for which computing and verifying ideal stable and accurate neural networks is extremely challenging.
  arXiv  Detail & Related papers  (2023-09-13T16:33:27Z)
• Neural Fields with Hard Constraints of Arbitrary Differential Order [61.49418682745144]
  We develop a series of approaches for enforcing hard constraints on neural fields. The constraints can be specified as a linear operator applied to the neural field and its derivatives. Our approaches are demonstrated in a wide range of real-world applications.
  arXiv  Detail & Related papers  (2023-06-15T08:33:52Z)
• Power Control with QoS Guarantees: A Differentiable Projection-based Unsupervised Learning Framework [14.518558523319518]
  Deep neural networks (DNNs) are emerging as a potential solution to solve NP-hard wireless resource allocation problems. We propose a novel unsupervised learning framework to solve the classical power control problem in a multi-user channel. We show that the proposed solutions not only improve the data rate but also achieve zero constraint violation probability, compared to the existing computations.
  arXiv  Detail & Related papers  (2023-05-31T14:11:51Z)
• DeepSaDe: Learning Neural Networks that Guarantee Domain Constraint Satisfaction [8.29487992932196]
  We present an approach to train neural networks which can enforce a wide variety of constraints and guarantee that the constraint is satisfied by all possible predictions. Our approach is flexible enough to enforce a wide variety of domain constraints and is able to guarantee them in neural networks.
  arXiv  Detail & Related papers  (2023-03-02T10:40:50Z)
• On the Forward Invariance of Neural ODEs [92.07281135902922]
  We propose a new method to ensure neural ordinary differential equations (ODEs) satisfy output specifications. Our approach uses a class of control barrier functions to transform output specifications into constraints on the parameters and inputs of the learning system.
  arXiv  Detail & Related papers  (2022-10-10T15:18:28Z)
• Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness [172.61581010141978]
  Certifiable robustness is a desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios. We propose a novel solution to strategically manipulate neurons, by "grafting" appropriate levels of linearity.
  arXiv  Detail & Related papers  (2022-06-15T22:42:29Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• A Kernel-Expanded Stochastic Neural Network [10.837308632004644]
  Deep neural network often gets trapped into a local minimum in training. New kernel-expanded neural network (K-StoNet) model reformulates the network as a latent variable model. Model can be easily trained using the imputationregularized optimization (IRO) algorithm.
  arXiv  Detail & Related papers  (2022-01-14T06:42:42Z)
• LQF: Linear Quadratic Fine-Tuning [114.3840147070712]
  We present the first method for linearizing a pre-trained model that achieves comparable performance to non-linear fine-tuning. LQF consists of simple modifications to the architecture, loss function and optimization typically used for classification.
  arXiv  Detail & Related papers  (2020-12-21T06:40:20Z)
• Attribute-Guided Adversarial Training for Robustness to Natural Perturbations [64.35805267250682]
  We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
  arXiv  Detail & Related papers  (2020-12-03T10:17:30Z)
• Chance-Constrained Control with Lexicographic Deep Reinforcement Learning [77.34726150561087]
  This paper proposes a lexicographic Deep Reinforcement Learning (DeepRL)-based approach to chance-constrained Markov Decision Processes. A lexicographic version of the well-known DeepRL algorithm DQN is also proposed and validated via simulations.
  arXiv  Detail & Related papers  (2020-10-19T13:09:14Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• Nonconvex sparse regularization for deep neural networks and its optimality [1.9798034349981162]
  Deep neural network (DNN) estimators can attain optimal convergence rates for regression and classification problems. We propose a novel penalized estimation method for sparse DNNs. We prove that the sparse-penalized estimator can adaptively attain minimax convergence rates for various nonparametric regression problems.
  arXiv  Detail & Related papers  (2020-03-26T07:15:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.