Learning With Multi-Group Guarantees For Clusterable Subpopulations

• URL: http://arxiv.org/abs/2410.14588v1
• Date: Fri, 18 Oct 2024 16:38:55 GMT
• Title: Learning With Multi-Group Guarantees For Clusterable Subpopulations
• Authors: Jessica Dai, Nika Haghtalab, Eric Zhao,
• Abstract summary: A canonical desideratum for prediction problems is that performance guarantees should hold on average over the population. But what constitutes a meaningful subpopulation? We take the perspective that relevant subpopulations should be defined with respect to the clusters that naturally emerge from the distribution of individuals.
• Score: 14.042643978487453
• License:
• Abstract: A canonical desideratum for prediction problems is that performance guarantees should hold not just on average over the population, but also for meaningful subpopulations within the overall population. But what constitutes a meaningful subpopulation? In this work, we take the perspective that relevant subpopulations should be defined with respect to the clusters that naturally emerge from the distribution of individuals for which predictions are being made. In this view, a population refers to a mixture model whose components constitute the relevant subpopulations. We suggest two formalisms for capturing per-subgroup guarantees: first, by attributing each individual to the component from which they were most likely drawn, given their features; and second, by attributing each individual to all components in proportion to their relative likelihood of having been drawn from each component. Using online calibration as a case study, we study a \variational algorithm that provides guarantees for each of these formalisms by handling all plausible underlying subpopulation structures simultaneously, and achieve an $O(T^{1/2})$ rate even when the subpopulations are not well-separated. In comparison, the more natural cluster-then-predict approach that first recovers the structure of the subpopulations and then makes predictions suffers from a $O(T^{2/3})$ rate and requires the subpopulations to be separable. Along the way, we prove that providing per-subgroup calibration guarantees for underlying clusters can be easier than learning the clusters: separation between median subgroup features is required for the latter but not the former.

Related papers

• A structured regression approach for evaluating model performance across intersectional subgroups [53.91682617836498]
  Disaggregated evaluation is a central task in AI fairness assessment, where the goal is to measure an AI system's performance across different subgroups. We introduce a structured regression approach to disaggregated evaluation that we demonstrate can yield reliable system performance estimates even for very small subgroups.
  arXiv  Detail & Related papers  (2024-01-26T14:21:45Z)
• Reweighted Mixup for Subpopulation Shift [63.1315456651771]
  Subpopulation shift exists in many real-world applications, which refers to the training and test distributions that contain the same subpopulation groups but with different subpopulation proportions. Importance reweighting is a classical and effective way to handle the subpopulation shift. We propose a simple yet practical framework, called reweighted mixup, to mitigate the overfitting issue.
  arXiv  Detail & Related papers  (2023-04-09T03:44:50Z)
• Modeling and Forecasting COVID-19 Cases using Latent Subpopulations [8.69240208462227]
  We propose two new methods to model the number of people infected with COVID-19 over time. Method #1 is a dictionary-based approach, which begins with a large number of pre-defined sub-population models. Method #2 is a mixture-of-$M$ fittable curves, where $M$, the number of sub-populations to use, is given by the user.
  arXiv  Detail & Related papers  (2023-02-09T18:33:41Z)
• KL Divergence Estimation with Multi-group Attribution [25.7757954754825]
  Estimating the Kullback-Leibler (KL) divergence between two distributions is well-studied in machine learning and information theory. Motivated by considerations of multi-group fairness, we seek KL divergence estimates that accurately reflect the contributions of sub-populations.
  arXiv  Detail & Related papers  (2022-02-28T06:54:10Z)
• Exact Recovery in the General Hypergraph Stochastic Block Model [92.28929858529679]
  This paper investigates fundamental limits of exact recovery in the general d-uniform hypergraph block model (d-HSBM) We show that there exists a sharp threshold such that exact recovery is achievable above the threshold and impossible below it.
  arXiv  Detail & Related papers  (2021-05-11T03:39:08Z)
• Robust subgroup discovery [0.2578242050187029]
  We formalize the problem of optimal robust subgroup discovery using the Minimum Description Length principle. We propose RSD, a greedy greedy that finds good subgroup lists and guarantees that the most significant subgroup is added in each iteration. We empirically show on 54 datasets that RSD outperforms previous subgroup set discovery methods in terms of quality and subgroup list size.
  arXiv  Detail & Related papers  (2021-03-25T09:04:13Z)
• Selective Classification Can Magnify Disparities Across Groups [89.14499988774985]
  We find that while selective classification can improve average accuracies, it can simultaneously magnify existing accuracy disparities. Increasing abstentions can even decrease accuracies on some groups. We train distributionally-robust models that achieve similar full-coverage accuracies across groups and show that selective classification uniformly improves each group.
  arXiv  Detail & Related papers  (2020-10-27T08:51:30Z)
• Structured Graph Learning for Clustering and Semi-supervised Classification [74.35376212789132]
  We propose a graph learning framework to preserve both the local and global structure of data. Our method uses the self-expressiveness of samples to capture the global structure and adaptive neighbor approach to respect the local structure. Our model is equivalent to a combination of kernel k-means and k-means methods under certain condition.
  arXiv  Detail & Related papers  (2020-08-31T08:41:20Z)
• Cumulative deviation of a subpopulation from the full population [0.0]
  Assessing equity in treatment of a subpopulation often involves assigning numerical "scores" to all individuals in the full population. Given such scores, individuals with similar scores may or may not attain similar outcomes independent of the individuals' memberships in the subpopulation. The cumulative plots encode subpopulation deviation directly as the slopes of secant lines for the graphs.
  arXiv  Detail & Related papers  (2020-08-04T19:30:02Z)
• Distributionally Robust Losses for Latent Covariate Mixtures [28.407773942857148]
  We propose a convex procedure that controls the worst-case performance over all subpopulations of a given size. We observe on lexical similarity, wine quality, and recidivism prediction tasks that our worst-case procedure learns models that do well against unseen subpopulations.
  arXiv  Detail & Related papers  (2020-07-28T04:16:27Z)
• Survival Cluster Analysis [93.50540270973927]
  There is an unmet need in survival analysis for identifying subpopulations with distinct risk profiles. An approach that addresses this need is likely to improve characterization of individual outcomes.
  arXiv  Detail & Related papers  (2020-02-29T22:41:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.