Multi-group Agnostic PAC Learnability

• URL: http://arxiv.org/abs/2105.09989v1
• Date: Thu, 20 May 2021 18:43:36 GMT
• Title: Multi-group Agnostic PAC Learnability
• Authors: Guy N Rothblum, Gal Yona
• Abstract summary: We study "multi-group agnostic PAC learnability" We provide a characterization of the loss functions for which such a predictor is guaranteed to exist. Our results unify and extend previous positive and negative results from the multi-group fairness literature.
• Score: 7.9649015115693444
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An agnostic PAC learning algorithm finds a predictor that is competitive with the best predictor in a benchmark hypothesis class, where competitiveness is measured with respect to a given loss function. However, its predictions might be quite sub-optimal for structured subgroups of individuals, such as protected demographic groups. Motivated by such fairness concerns, we study "multi-group agnostic PAC learnability": fixing a measure of loss, a benchmark class $\H$ and a (potentially) rich collection of subgroups $\G$, the objective is to learn a single predictor such that the loss experienced by every group $g \in \G$ is not much larger than the best possible loss for this group within $\H$. Under natural conditions, we provide a characterization of the loss functions for which such a predictor is guaranteed to exist. For any such loss function we construct a learning algorithm whose sample complexity is logarithmic in the size of the collection $\G$. Our results unify and extend previous positive and negative results from the multi-group fairness literature, which applied for specific loss functions.

Related papers

• Top-$k$ Classification and Cardinality-Aware Prediction [30.389055604165222]
  We show that comp-sum and constrained losses are supported by $H$-consistency bounds with respect to the top-$k$ loss. We introduce cardinality-aware loss functions through instance-dependent cost-sensitive learning. Minimizing these losses leads to new cardinality-aware algorithms for top-$k$ classification.
  arXiv  Detail & Related papers  (2024-03-28T17:45:03Z)
• Fair Active Ranking from Pairwise Preferences [6.102498508368527]
  Given a set of $n$ items that belong to disjoint groups, our goal is to find an $(epsilon, delta)$-PACF-Ranking according to a fair objective function that we propose. We present both group-blind and group-aware algorithms and analyze their sample parameters.
  arXiv  Detail & Related papers  (2024-02-05T18:09:48Z)
• A structured regression approach for evaluating model performance across intersectional subgroups [53.91682617836498]
  Disaggregated evaluation is a central task in AI fairness assessment, where the goal is to measure an AI system's performance across different subgroups. We introduce a structured regression approach to disaggregated evaluation that we demonstrate can yield reliable system performance estimates even for very small subgroups.
  arXiv  Detail & Related papers  (2024-01-26T14:21:45Z)
• Agnostically Learning Multi-index Models with Queries [54.290489524576756]
  We study the power of query access for the task of agnostic learning under the Gaussian distribution. We show that query access gives significant runtime improvements over random examples for agnostically learning MIMs.
  arXiv  Detail & Related papers  (2023-12-27T15:50:47Z)
• Agnostic Multi-Group Active Learning [24.97598179536084]
  We consider a variant of this problem from the perspective of active learning, where the learner is endowed with the power to decide which examples are labeled from each distribution in the collection. Our main challenge is that standard active learning techniques such as disagreement-based active learning do not directly apply to the multi-group learning objective. We modify existing algorithms to provide a consistent active learning algorithm for an agnostic formulation of multi-group learning.
  arXiv  Detail & Related papers  (2023-06-02T21:24:13Z)
• Characterizing the Optimal 0-1 Loss for Multi-class Classification with a Test-time Attacker [57.49330031751386]
  We find achievable information-theoretic lower bounds on loss in the presence of a test-time attacker for multi-class classifiers on any discrete dataset. We provide a general framework for finding the optimal 0-1 loss that revolves around the construction of a conflict hypergraph from the data and adversarial constraints.
  arXiv  Detail & Related papers  (2023-02-21T15:17:13Z)
• The Group Loss++: A deeper look into group loss for deep metric learning [65.19665861268574]
  Group Loss is a loss function based on a differentiable label-propagation method that enforces embedding similarity across all samples of a group. We show state-of-the-art results on clustering and image retrieval on four datasets, and present competitive results on two person re-identification datasets.
  arXiv  Detail & Related papers  (2022-04-04T14:09:58Z)
• Unified Negative Pair Generation toward Well-discriminative Feature Space for Face Recognition [2.816374336026564]
  Face recognition models form a well-discriminative feature space (WDFS) that satisfies $infmathcalSp > supmathcalSn$. This paper proposes a unified negative pair generation (UNPG) by combining two PG strategies.
  arXiv  Detail & Related papers  (2022-03-22T10:21:11Z)
• Omnipredictors [19.735769148626588]
  Loss minimization is a dominant paradigm in machine learning. We introduce the notion of an ($mathcalL,mathcalC$)-omnipredictor, which could be used to optimize any loss in a family. We show that such "loss-oblivious'' learning is feasible through a connection to multicalibration.
  arXiv  Detail & Related papers  (2021-09-11T23:28:49Z)
• Examining and Combating Spurious Features under Distribution Shift [94.31956965507085]
  We define and analyze robust and spurious representations using the information-theoretic concept of minimal sufficient statistics. We prove that even when there is only bias of the input distribution, models can still pick up spurious features from their training data. Inspired by our analysis, we demonstrate that group DRO can fail when groups do not directly account for various spurious correlations.
  arXiv  Detail & Related papers  (2021-06-14T05:39:09Z)
• Learning by Minimizing the Sum of Ranked Range [58.24935359348289]
  We introduce the sum of ranked range (SoRR) as a general approach to form learning objectives. A ranked range is a consecutive sequence of sorted values of a set of real numbers. We explore two applications in machine learning of the minimization of the SoRR framework, namely the AoRR aggregate loss for binary classification and the TKML individual loss for multi-label/multi-class classification.
  arXiv  Detail & Related papers  (2020-10-05T01:58:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.