PAC-Bayesian Bounds on Constrained f-Entropic Risk Measures

• URL: http://arxiv.org/abs/2510.11169v1
• Date: Mon, 13 Oct 2025 09:02:13 GMT
• Title: PAC-Bayesian Bounds on Constrained f-Entropic Risk Measures
• Authors: Hind Atbir, Farah Cherfaoui, Guillaume Metzler, Emilie Morvant, Paul Viallard,
• Abstract summary: We introduce constrained f-entropic risk measures, which enable finer control over distributional shifts and subgroup imbalances via f-divergences.<n>We derive both classical and disintegrated PAC-Bayesian generalization bounds for this family of risks.<n>We design a self-bounding algorithm that minimizes our bounds directly, yielding models with guarantees at the subgroup level.
• Score: 3.8806557528413292
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: PAC generalization bounds on the risk, when expressed in terms of the expected loss, are often insufficient to capture imbalances between subgroups in the data. To overcome this limitation, we introduce a new family of risk measures, called constrained f-entropic risk measures, which enable finer control over distributional shifts and subgroup imbalances via f-divergences, and include the Conditional Value at Risk (CVaR), a well-known risk measure. We derive both classical and disintegrated PAC-Bayesian generalization bounds for this family of risks, providing the first disintegratedPAC-Bayesian guarantees beyond standard risks. Building on this theory, we design a self-bounding algorithm that minimizes our bounds directly, yielding models with guarantees at the subgroup level. Finally, we empirically demonstrate the usefulness of our approach.

Related papers

• PAC-Bayesian Generalization Guarantees for Fairness on Stochastic and Deterministic Classifiers [8.438034474012044]
  We propose a PAC-Bayesian framework for deriving generalization bounds for fairness.<n>Our framework has two advantages: (i) It applies to a broad class of fairness measures that can be expressed as a risk discrepancy, and (ii) it leads to a self-bounding algorithm.
  arXiv  Detail & Related papers  (2026-02-12T08:49:34Z)
• Constrained Language Model Policy Optimization via Risk-aware Stepwise Alignment [49.2305683068875]
  We propose Risk-aware Stepwise Alignment (RSA), a novel alignment method that incorporates risk awareness into the policy optimization process.<n> RSA mitigates risks induced by excessive model shift away from a reference policy, and it explicitly suppresses low-probability yet high-impact harmful behaviors.<n> Experimental results demonstrate that our method achieves high levels of helpfulness while ensuring strong safety.
  arXiv  Detail & Related papers  (2025-12-30T14:38:02Z)
• Risk-Averse Constrained Reinforcement Learning with Optimized Certainty Equivalents [29.698100324454362]
  Constrained optimization provides a common framework for dealing with conflicting objectives in reinforcement learning (RL)<n>We propose a framework for risk-aware constrained RL, which exhibits per-stage properties jointly in reward values and time using optimized certainty equivalents (OCEs)<n>Our framework ensures an exact equivalent to the original constrained problem within a parameterized strong Lagrangian duality framework under appropriate constraint qualifications.
  arXiv  Detail & Related papers  (2025-10-23T04:33:32Z)
• From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks [54.25638567385662]
  Differential Privacy in machine learning is often interpreted as guarantees against membership inference.<n> translating DP budgets into quantitative protection against the more damaging threat of data reconstruction remains a challenging open problem.<n>This paper bridges the critical gap by deriving the first formal privacy bounds tailored to the mechanics of demonstrated "from-scratch" attacks.
  arXiv  Detail & Related papers  (2024-02-20T09:52:30Z)
• Strong Duality Relations in Nonconvex Risk-Constrained Learning [0.0]
  J. J. Uhl's convexity for general infinite dimensional Banach spaces is an extension of A. A. Lynovs. We show that constrained classification and regression can be treated under a unifying lens, while certain restrictive assumptions enforced in the current literature are a new state of the art.
  arXiv  Detail & Related papers  (2023-12-02T11:21:00Z)
• Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified Models [9.65010022854885]
  We show that adversarial risk is equivalent to the risk induced by a distributional adversarial attack under certain smoothness conditions. To evaluate the generalization performance of the adversarial estimator, we study the adversarial excess risk.
  arXiv  Detail & Related papers  (2023-09-02T00:51:19Z)
• Domain Generalization without Excess Empirical Risk [83.26052467843725]
  A common approach is designing a data-driven surrogate penalty to capture generalization and minimize the empirical risk jointly with the penalty. We argue that a significant failure mode of this recipe is an excess risk due to an erroneous penalty or hardness in joint optimization. We present an approach that eliminates this problem. Instead of jointly minimizing empirical risk with the penalty, we minimize the penalty under the constraint of optimality of the empirical risk.
  arXiv  Detail & Related papers  (2023-08-30T08:46:46Z)
• Capsa: A Unified Framework for Quantifying Risk in Deep Neural Networks [142.67349734180445]
  Existing algorithms that provide risk-awareness to deep neural networks are complex and ad-hoc. Here we present capsa, a framework for extending models with risk-awareness.
  arXiv  Detail & Related papers  (2023-08-01T02:07:47Z)
• Risk-Averse Reinforcement Learning via Dynamic Time-Consistent Risk Measures [10.221369785560785]
  In this paper, we consider the problem of maximizing dynamic risk of a sequence of rewards in Markov Decision Processes (MDPs) Using a convex combination of expectation and conditional value-at-risk (CVaR) as a special one-step conditional risk measure, we reformulate the risk-averse MDP as a risk-neutral counterpart with augmented action space and manipulation on the immediate rewards. Our numerical studies show that the risk-averse setting can reduce the variance and enhance robustness of the results.
  arXiv  Detail & Related papers  (2023-01-14T21:43:18Z)
• On the Importance of Gradient Norm in PAC-Bayesian Bounds [92.82627080794491]
  We propose a new generalization bound that exploits the contractivity of the log-Sobolev inequalities. We empirically analyze the effect of this new loss-gradient norm term on different neural architectures.
  arXiv  Detail & Related papers  (2022-10-12T12:49:20Z)
• Risk Aware Belief-dependent Constrained POMDP Planning [9.061408029414453]
  Risk awareness is fundamental to an online operating agent. Existing constrained POMDP algorithms are typically designed for discrete state and observation spaces. This paper presents a novel formulation for risk-averse belief-dependent constrained POMDP.
  arXiv  Detail & Related papers  (2022-09-06T17:48:13Z)
• Supervised Learning with General Risk Functionals [28.918233583859134]
  Standard uniform convergence results bound the generalization gap of the expected loss over a hypothesis class. We establish the first uniform convergence results for estimating the CDF of the loss distribution, yielding guarantees that hold simultaneously both over all H"older risk functionals and over all hypotheses.
  arXiv  Detail & Related papers  (2022-06-27T22:11:05Z)
• Risk-Constrained Thompson Sampling for CVaR Bandits [82.47796318548306]
  We consider a popular risk measure in quantitative finance known as the Conditional Value at Risk (CVaR) We explore the performance of a Thompson Sampling-based algorithm CVaR-TS under this risk measure.
  arXiv  Detail & Related papers  (2020-11-16T15:53:22Z)
• Learning Bounds for Risk-sensitive Learning [86.50262971918276]
  In risk-sensitive learning, one aims to find a hypothesis that minimizes a risk-averse (or risk-seeking) measure of loss. We study the generalization properties of risk-sensitive learning schemes whose optimand is described via optimized certainty equivalents.
  arXiv  Detail & Related papers  (2020-06-15T05:25:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.