End-to-End Probabilistic Framework for Learning with Hard Constraints

• URL: http://arxiv.org/abs/2506.07003v1
• Date: Sun, 08 Jun 2025 05:29:50 GMT
• Title: End-to-End Probabilistic Framework for Learning with Hard Constraints
• Authors: Utkarsh Utkarsh, Danielle C. Maddix, Ruijun Ma, Michael W. Mahoney, Yuyang Wang,
• Abstract summary: ProbHardE2E learns systems that can incorporate operational/physical constraints as hard requirements.<n>It enforces hard constraints by exploiting variance information in a novel way.<n>It can incorporate a range of non-linear constraints (increasing the power of modeling and flexibility)
• Score: 47.10876360975842
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a general purpose probabilistic forecasting framework, ProbHardE2E, to learn systems that can incorporate operational/physical constraints as hard requirements. ProbHardE2E enforces hard constraints by exploiting variance information in a novel way; and thus it is also capable of performing uncertainty quantification (UQ) on the model. Our methodology uses a novel differentiable probabilistic projection layer (DPPL) that can be combined with a wide range of neural network architectures. This DPPL allows the model to learn the system in an end-to-end manner, compared to other approaches where the constraints are satisfied either through a post-processing step or at inference. In addition, ProbHardE2E can optimize a strictly proper scoring rule, without making any distributional assumptions on the target, which enables it to obtain robust distributional estimates (in contrast to existing approaches that generally optimize likelihood-based objectives, which are heavily biased by their distributional assumptions and model choices); and it can incorporate a range of non-linear constraints (increasing the power of modeling and flexibility). We apply ProbHardE2E to problems in learning partial differential equations with uncertainty estimates and to probabilistic time-series forecasting, showcasing it as a broadly applicable general setup that connects these seemingly disparate domains.

Related papers

• LAPO: Internalizing Reasoning Efficiency via Length-Adaptive Policy Optimization [48.91511514636768]
  We present Length-Adaptive Policy Optimization (LAPO), a framework that transforms reasoning length control from an external constraint into an intrinsic model capability.<n>LAPO enables models to internalize an understanding of appropriate reasoning depth through a two-stage reinforcement learning process.<n> Experiments on mathematical reasoning benchmarks demonstrate that LAPO reduces token usage by up to 40.9% while improving accuracy by 2.3%.
  arXiv  Detail & Related papers  (2025-07-21T16:14:41Z)
• Enforcing Hard Linear Constraints in Deep Learning Models with Decision Rules [8.098452803458253]
  This paper proposes a model-agnostic framework for enforcing input-dependent linear equality and inequality constraints on neural network outputs.<n>The architecture combines a task network trained for prediction accuracy with a safe network trained using decision rules from the runtime and robust optimization to ensure feasibility across the entire input space.
  arXiv  Detail & Related papers  (2025-05-20T03:09:44Z)
• Probabilistic neural operators for functional uncertainty quantification [14.08907045605149]
  We introduce the probabilistic neural operator (PNO), a framework for learning probability distributions over the output function space of neural operators.<n>PNO extends neural operators with generative modeling based on strictly proper scoring rules, integrating uncertainty information directly into the training process.
  arXiv  Detail & Related papers  (2025-02-18T14:42:11Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• Collaborative Uncertainty Benefits Multi-Agent Multi-Modal Trajectory Forecasting [61.02295959343446]
  This work first proposes a novel concept, collaborative uncertainty (CU), which models the uncertainty resulting from interaction modules.<n>We build a general CU-aware regression framework with an original permutation-equivariant uncertainty estimator to do both tasks of regression and uncertainty estimation.<n>We apply the proposed framework to current SOTA multi-agent trajectory forecasting systems as a plugin module.
  arXiv  Detail & Related papers  (2022-07-11T21:17:41Z)
• Semantic Probabilistic Layers for Neuro-Symbolic Learning [83.25785999205932]
  We design a predictive layer for structured-output prediction (SOP) It can be plugged into any neural network guaranteeing its predictions are consistent with a set of predefined symbolic constraints. Our Semantic Probabilistic Layer (SPL) can model intricate correlations, and hard constraints, over a structured output space.
  arXiv  Detail & Related papers  (2022-06-01T12:02:38Z)
• Modeling the Second Player in Distributionally Robust Optimization [90.25995710696425]
  We argue for the use of neural generative models to characterize the worst-case distribution. This approach poses a number of implementation and optimization challenges. We find that the proposed approach yields models that are more robust than comparable baselines.
  arXiv  Detail & Related papers  (2021-03-18T14:26:26Z)
• Probabilistic electric load forecasting through Bayesian Mixture Density Networks [70.50488907591463]
  Probabilistic load forecasting (PLF) is a key component in the extended tool-chain required for efficient management of smart energy grids. We propose a novel PLF approach, framed on Bayesian Mixture Density Networks. To achieve reliable and computationally scalable estimators of the posterior distributions, both Mean Field variational inference and deep ensembles are integrated.
  arXiv  Detail & Related papers  (2020-12-23T16:21:34Z)
• Deep Conditional Transformation Models [0.0]
  Learning the cumulative distribution function (CDF) of an outcome variable conditional on a set of features remains challenging. Conditional transformation models provide a semi-parametric approach that allows to model a large class of conditional CDFs. We propose a novel network architecture, provide details on different model definitions and derive suitable constraints.
  arXiv  Detail & Related papers  (2020-10-15T16:25:45Z)
• Robust-Adaptive Control of Linear Systems: beyond Quadratic Costs [14.309243378538012]
  We consider the problem of robust and adaptive model predictive control (MPC) of a linear system. We provide the first end-to-end suboptimal tractity analysis for this setting.
  arXiv  Detail & Related papers  (2020-02-25T12:24:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.