Conditional Risk Minimization with Side Information: A Tractable, Universal Optimal Transport Framework

• URL: http://arxiv.org/abs/2509.23128v1
• Date: Sat, 27 Sep 2025 05:22:53 GMT
• Title: Conditional Risk Minimization with Side Information: A Tractable, Universal Optimal Transport Framework
• Authors: Xinqiao Xie, Jonathan Yu-Meng Li,
• Abstract summary: Conditional risk minimization arises in high-stakes decisions where risk must be assessed in light of side information.<n>We introduce a universal framework for distributionally robust conditional risk minimization, built on a novel union-ball formulation in optimal transport.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conditional risk minimization arises in high-stakes decisions where risk must be assessed in light of side information, such as stressed economic conditions, specific customer profiles, or other contextual covariates. Constructing reliable conditional distributions from limited data is notoriously difficult, motivating a series of optimal-transport-based proposals that address this uncertainty in a distributionally robust manner. Yet these approaches remain fragmented, each constrained by its own limitations: some rely on point estimates or restrictive structural assumptions, others apply only to narrow classes of risk measures, and their structural connections are unclear. We introduce a universal framework for distributionally robust conditional risk minimization, built on a novel union-ball formulation in optimal transport. This framework offers three key advantages: interpretability, by subsuming existing methods as special cases and revealing their deep structural links; tractability, by yielding convex reformulations for virtually all major risk functionals studied in the literature; and scalability, by supporting cutting-plane algorithms for large-scale conditional risk problems. Applications to portfolio optimization with rank-dependent expected utility highlight the practical effectiveness of the framework, with conditional models converging to optimal solutions where unconditional ones clearly do not.

Related papers

• Constrained Language Model Policy Optimization via Risk-aware Stepwise Alignment [49.2305683068875]
  We propose Risk-aware Stepwise Alignment (RSA), a novel alignment method that incorporates risk awareness into the policy optimization process.<n> RSA mitigates risks induced by excessive model shift away from a reference policy, and it explicitly suppresses low-probability yet high-impact harmful behaviors.<n> Experimental results demonstrate that our method achieves high levels of helpfulness while ensuring strong safety.
  arXiv  Detail & Related papers  (2025-12-30T14:38:02Z)
• Adaptive Neighborhood-Constrained Q Learning for Offline Reinforcement Learning [52.03884701766989]
  offline reinforcement learning (RL) algorithms typically impose constraints on action selection.<n>We propose a new neighborhood constraint that restricts action selection in the Bellman target to the union of neighborhoods of dataset actions.<n>We develop a simple yet effective algorithm, Adaptive Neighborhood-constrained Q learning (ANQ), to perform Q learning with target actions satisfying this constraint.
  arXiv  Detail & Related papers  (2025-11-04T13:42:05Z)
• Risk-Averse Constrained Reinforcement Learning with Optimized Certainty Equivalents [29.698100324454362]
  Constrained optimization provides a common framework for dealing with conflicting objectives in reinforcement learning (RL)<n>We propose a framework for risk-aware constrained RL, which exhibits per-stage properties jointly in reward values and time using optimized certainty equivalents (OCEs)<n>Our framework ensures an exact equivalent to the original constrained problem within a parameterized strong Lagrangian duality framework under appropriate constraint qualifications.
  arXiv  Detail & Related papers  (2025-10-23T04:33:32Z)
• Rectified Robust Policy Optimization for Model-Uncertain Constrained Reinforcement Learning without Strong Duality [53.525547349715595]
  We propose a novel primal-only algorithm called Rectified Robust Policy Optimization (RRPO)<n>RRPO operates directly on the primal problem without relying on dual formulations.<n>We show convergence to an approximately optimal feasible policy with complexity matching the best-known lower bound.
  arXiv  Detail & Related papers  (2025-08-24T16:59:38Z)
• SConU: Selective Conformal Uncertainty in Large Language Models [59.25881667640868]
  We propose a novel approach termed Selective Conformal Uncertainty (SConU)<n>We develop two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level.<n>Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions.
  arXiv  Detail & Related papers  (2025-04-19T03:01:45Z)
• Achievable distributional robustness when the robust risk is only partially identified [8.192907805418583]
  In safety-critical applications, machine learning models should generalize well under worst-case distribution shifts.<n>We introduce the worst-case robust risk as a new measure of robustness that is always well-defined regardless of identifiability.<n>First, we show that existing robustness methods are provably suboptimal in the partially identifiable case.
  arXiv  Detail & Related papers  (2025-02-04T20:42:47Z)
• Fair Risk Minimization under Causal Path-Specific Effect Constraints [3.0232957374216953]
  This paper introduces a framework for estimating fair optimal predictions using machine learning. We derive closed-form solutions for constrained optimization based on mean squared error and cross-entropy risk criteria.
  arXiv  Detail & Related papers  (2024-08-03T02:05:43Z)
• Generalization Bounds of Surrogate Policies for Combinatorial Optimization Problems [61.580419063416734]
  A recent stream of structured learning approaches has improved the practical state of the art for a range of optimization problems. The key idea is to exploit the statistical distribution over instances instead of dealing with instances separately. In this article, we investigate methods that smooth the risk by perturbing the policy, which eases optimization and improves the generalization error.
  arXiv  Detail & Related papers  (2024-07-24T12:00:30Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Statistical learning for constrained functional parameters in infinite-dimensional models [4.974815773537217]
  We develop a framework for estimating function-valued parameters under equality or inequality constraints in infinite-dimensional statistical models.<n>We characterize the solution as the minimizer of a penalized population risk using a Lagrange-type formulation.<n>We demonstrate the flexibility and effectiveness of the proposed method through various examples, simulations, and real-data applications.
  arXiv  Detail & Related papers  (2024-04-15T14:59:21Z)
• Domain Generalization without Excess Empirical Risk [83.26052467843725]
  A common approach is designing a data-driven surrogate penalty to capture generalization and minimize the empirical risk jointly with the penalty. We argue that a significant failure mode of this recipe is an excess risk due to an erroneous penalty or hardness in joint optimization. We present an approach that eliminates this problem. Instead of jointly minimizing empirical risk with the penalty, we minimize the penalty under the constraint of optimality of the empirical risk.
  arXiv  Detail & Related papers  (2023-08-30T08:46:46Z)
• Out-of-Distribution Optimality of Invariant Risk Minimization [20.389816785165333]
  Invariant Risk Minimization (IRM) is considered to be a promising approach to minimize the o.o.d. risk. This paper rigorously proves that a solution to the bi-level optimization problem minimizes the o.o.d. risk under certain conditions.
  arXiv  Detail & Related papers  (2023-07-22T03:31:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.