Group conditional validity via multi-group learning

• URL: http://arxiv.org/abs/2303.03995v2
• Date: Sun, 19 Mar 2023 17:17:11 GMT
• Title: Group conditional validity via multi-group learning
• Authors: Samuel Deng, Navid Ardeshir, Daniel Hsu
• Abstract summary: We consider the problem of distribution-free conformal prediction and the criterion of group conditional validity. Existing methods achieve such guarantees under either restrictive grouping structure or distributional assumptions. We propose a simple reduction to the problem of achieving validity guarantees for individual populations by leveraging algorithms for a problem called multi-group learning.
• Score: 5.797821810358083
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of distribution-free conformal prediction and the criterion of group conditional validity. This criterion is motivated by many practical scenarios including hidden stratification and group fairness. Existing methods achieve such guarantees under either restrictive grouping structure or distributional assumptions, or they are overly-conservative under heteroskedastic noise. We propose a simple reduction to the problem of achieving validity guarantees for individual populations by leveraging algorithms for a problem called multi-group learning. This allows us to port theoretical guarantees from multi-group learning to obtain obtain sample complexity guarantees for conformal prediction. We also provide a new algorithm for multi-group learning for groups with hierarchical structure. Using this algorithm in our reduction leads to improved sample complexity guarantees with a simpler predictor structure.

Related papers

• Finite-Sample and Distribution-Free Fair Classification: Optimal Trade-off Between Excess Risk and Fairness, and the Cost of Group-Blindness [14.421493372559762]
  We quantify the impact of enforcing algorithmic fairness and group-blindness in binary classification under group fairness constraints. We propose a unified framework for fair classification that provides distribution-free and finite-sample fairness guarantees with controlled excess risk.
  arXiv  Detail & Related papers  (2024-10-21T20:04:17Z)
• Structured Conformal Inference for Matrix Completion with Applications to Group Recommender Systems [16.519348575982004]
  We develop a conformal inference method to construct joint confidence regions for structured groups of missing entries. Our method achieves stronger group-level guarantees by carefully assembling a structured calibration data set.
  arXiv  Detail & Related papers  (2024-04-26T17:42:29Z)
• How does promoting the minority fraction affect generalization? A theoretical study of the one-hidden-layer neural network on group imbalance [64.1656365676171]
  Group imbalance has been a known problem in empirical risk minimization. This paper quantifies the impact of individual groups on the sample complexity, the convergence rate, and the average and group-level testing performance.
  arXiv  Detail & Related papers  (2024-03-12T04:38:05Z)
• Multi-group Learning for Hierarchical Groups [12.473780585666768]
  We extend the study of multi-group learning to the natural case where the groups are hierarchically structured. We design an algorithm for this setting that outputs an interpretable and deterministic decision tree predictor with near-optimal sample complexity.
  arXiv  Detail & Related papers  (2024-02-01T01:06:32Z)
• A structured regression approach for evaluating model performance across intersectional subgroups [53.91682617836498]
  Disaggregated evaluation is a central task in AI fairness assessment, where the goal is to measure an AI system's performance across different subgroups. We introduce a structured regression approach to disaggregated evaluation that we demonstrate can yield reliable system performance estimates even for very small subgroups.
  arXiv  Detail & Related papers  (2024-01-26T14:21:45Z)
• Optimal Multi-Distribution Learning [88.3008613028333]
  Multi-distribution learning seeks to learn a shared model that minimizes the worst-case risk across $k$ distinct data distributions. We propose a novel algorithm that yields an varepsilon-optimal randomized hypothesis with a sample complexity on the order of (d+k)/varepsilon2.
  arXiv  Detail & Related papers  (2023-12-08T16:06:29Z)
• Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning [79.83792914684985]
  We prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem.
  arXiv  Detail & Related papers  (2022-11-26T21:02:09Z)
• Simple and near-optimal algorithms for hidden stratification and multi-group learning [13.337579367787253]
  This paper studies the structure of solutions to the multi-group learning problem. It provides simple and near-optimal algorithms for the learning problem.
  arXiv  Detail & Related papers  (2021-12-22T19:16:24Z)
• Group Testing with Non-identical Infection Probabilities [59.96266198512243]
  We develop an adaptive group testing algorithm using the set formation method. We show that our algorithm outperforms the state of the art, and performs close to the entropy lower bound.
  arXiv  Detail & Related papers  (2021-08-27T17:53:25Z)
• Exclusive Group Lasso for Structured Variable Selection [10.86544864007391]
  A structured variable selection problem is considered. A composite norm can be properly designed to promote such exclusive group sparsity patterns. An active set algorithm is proposed that builds the solution by including structure atoms into the estimated support.
  arXiv  Detail & Related papers  (2021-08-23T16:55:13Z)
• Robust Optimization for Fairness with Noisy Protected Groups [85.13255550021495]
  We study the consequences of naively relying on noisy protected group labels. We introduce two new approaches using robust optimization. We show that the robust approaches achieve better true group fairness guarantees than the naive approach.
  arXiv  Detail & Related papers  (2020-02-21T14:58:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.