To Split or Not to Split: The Impact of Disparate Treatment in Classification

• URL: http://arxiv.org/abs/2002.04788v4
• Date: Thu, 14 Apr 2022 01:20:49 GMT
• Title: To Split or Not to Split: The Impact of Disparate Treatment in Classification
• Authors: Hao Wang, Hsiang Hsu, Mario Diaz, Flavio P. Calmon
• Abstract summary: Disparate treatment occurs when a machine learning model yields different decisions for individuals based on a sensitive attribute. We introduce the benefit-of-splitting for quantifying the performance improvement by splitting classifiers. We prove an equivalent expression for the benefit-of-splitting which can be efficiently computed by solving small-scale convex programs.
• Score: 8.325775867295814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Disparate treatment occurs when a machine learning model yields different decisions for individuals based on a sensitive attribute (e.g., age, sex). In domains where prediction accuracy is paramount, it could potentially be acceptable to fit a model which exhibits disparate treatment. To evaluate the effect of disparate treatment, we compare the performance of split classifiers (i.e., classifiers trained and deployed separately on each group) with group-blind classifiers (i.e., classifiers which do not use a sensitive attribute). We introduce the benefit-of-splitting for quantifying the performance improvement by splitting classifiers. Computing the benefit-of-splitting directly from its definition could be intractable since it involves solving optimization problems over an infinite-dimensional functional space. Under different performance measures, we (i) prove an equivalent expression for the benefit-of-splitting which can be efficiently computed by solving small-scale convex programs; (ii) provide sharp upper and lower bounds for the benefit-of-splitting which reveal precise conditions where a group-blind classifier will always suffer from a non-trivial performance gap from the split classifiers. In the finite sample regime, splitting is not necessarily beneficial and we provide data-dependent bounds to understand this effect. Finally, we validate our theoretical results through numerical experiments on both synthetic and real-world datasets.

Related papers

• Probabilistic Contrastive Learning for Long-Tailed Visual Recognition [78.70453964041718]
  Longtailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples. Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance. We propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space.
  arXiv  Detail & Related papers  (2024-03-11T13:44:49Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Extension of Transformational Machine Learning: Classification Problems [0.0]
  This study explores the application and performance of Transformational Machine Learning (TML) in drug discovery. TML, a meta learning algorithm, excels in exploiting common attributes across various domains. The drug discovery process, which is complex and time-consuming, can benefit greatly from the enhanced prediction accuracy.
  arXiv  Detail & Related papers  (2023-08-07T07:34:18Z)
• Malign Overfitting: Interpolation Can Provably Preclude Invariance [30.776243638012314]
  We show that "benign overfitting" in which models generalize well despite interpolating might not favorably extend to settings in which robustness or fairness are desirable. We propose and analyze an algorithm that successfully learns a non-interpolating classifier that is provably invariant.
  arXiv  Detail & Related papers  (2022-11-28T19:17:31Z)
• Fine-grained Retrieval Prompt Tuning [149.9071858259279]
  Fine-grained Retrieval Prompt Tuning steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompt and feature adaptation. Our FRPT with fewer learnable parameters achieves the state-of-the-art performance on three widely-used fine-grained datasets.
  arXiv  Detail & Related papers  (2022-07-29T04:10:04Z)
• The Impact of Using Regression Models to Build Defect Classifiers [13.840006058766766]
  It is common practice to discretize continuous defect counts into defective and non-defective classes. We compare the performance and interpretation of defect classifiers built using both approaches.
  arXiv  Detail & Related papers  (2022-02-12T22:12:55Z)
• How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers [0.1611401281366893]
  In conformal classification, the systems can output multiple class labels instead of one. For a Neural Network-based conformal classifier, the inverse probability allows minimizing the average number of predicted labels. We propose a successful method to combine the properties of these two nonconformity functions.
  arXiv  Detail & Related papers  (2021-08-12T11:50:12Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• Asymptotic Supervised Predictive Classifiers under Partition Exchangeability [0.0]
  The result shows the convergence of these classifiers under infinite amount of training or test data. This is an important result from the practical perspective as under the presence of sufficiently large amount of data, one can replace the simpler marginal classifier with computationally more expensive simultaneous one.
  arXiv  Detail & Related papers  (2021-01-26T17:17:40Z)
• Rethinking preventing class-collapsing in metric learning with margin-based losses [81.22825616879936]
  Metric learning seeks embeddings where visually similar instances are close and dissimilar instances are apart. margin-based losses tend to project all samples of a class onto a single point in the embedding space. We propose a simple modification to the embedding losses such that each sample selects its nearest same-class counterpart in a batch.
  arXiv  Detail & Related papers  (2020-06-09T09:59:25Z)
• Almost-Matching-Exactly for Treatment Effect Estimation under Network Interference [73.23326654892963]
  We propose a matching method that recovers direct treatment effects from randomized experiments where units are connected in an observed network. Our method matches units almost exactly on counts of unique subgraphs within their neighborhood graphs.
  arXiv  Detail & Related papers  (2020-03-02T15:21:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.