AutoBalance: Optimized Loss Functions for Imbalanced Data
- URL: http://arxiv.org/abs/2201.01212v1
- Date: Tue, 4 Jan 2022 15:53:23 GMT
- Title: AutoBalance: Optimized Loss Functions for Imbalanced Data
- Authors: Mingchen Li, Xuechen Zhang, Christos Thrampoulidis, Jiasi Chen, Samet
Oymak
- Abstract summary: We propose AutoBalance, a bi-level optimization framework that automatically designs a training loss function to optimize a blend of accuracy and fairness-seeking objectives.
Specifically, a lower-level problem trains the model weights, and an upper-level problem tunes the loss function by monitoring and optimizing the desired objective over the validation data.
Our loss design enables personalized treatment for classes/groups by employing a parametric cross-entropy loss and individualized data augmentation schemes.
- Score: 38.64606886588534
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Imbalanced datasets are commonplace in modern machine learning problems. The
presence of under-represented classes or groups with sensitive attributes
results in concerns about generalization and fairness. Such concerns are
further exacerbated by the fact that large capacity deep nets can perfectly fit
the training data and appear to achieve perfect accuracy and fairness during
training, but perform poorly during test. To address these challenges, we
propose AutoBalance, a bi-level optimization framework that automatically
designs a training loss function to optimize a blend of accuracy and
fairness-seeking objectives. Specifically, a lower-level problem trains the
model weights, and an upper-level problem tunes the loss function by monitoring
and optimizing the desired objective over the validation data. Our loss design
enables personalized treatment for classes/groups by employing a parametric
cross-entropy loss and individualized data augmentation schemes. We evaluate
the benefits and performance of our approach for the application scenarios of
imbalanced and group-sensitive classification. Extensive empirical evaluations
demonstrate the benefits of AutoBalance over state-of-the-art approaches. Our
experimental findings are complemented with theoretical insights on loss
function design and the benefits of train-validation split. All code is
available open-source.
Related papers
- Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium [0.3350491650545292]
Current methods for mitigating bias often result in information loss and an inadequate balance between accuracy and fairness.
We propose a novel methodology grounded in bilevel optimization principles.
Our deep learning-based approach concurrently optimize for both accuracy and fairness objectives.
arXiv Detail & Related papers (2024-10-21T18:53:39Z) - Optimizing for ROC Curves on Class-Imbalanced Data by Training over a Family of Loss Functions [3.06506506650274]
Training reliable classifiers under severe class imbalance is a challenging problem in computer vision.
Recent work has proposed techniques that mitigate the effects of training under imbalance by modifying the loss functions or optimization methods.
We propose training over a family of loss functions, instead of a single loss function.
arXiv Detail & Related papers (2024-02-08T04:31:21Z) - Simplifying Neural Network Training Under Class Imbalance [77.39968702907817]
Real-world datasets are often highly class-imbalanced, which can adversely impact the performance of deep learning models.
The majority of research on training neural networks under class imbalance has focused on specialized loss functions, sampling techniques, or two-stage training procedures.
We demonstrate that simply tuning existing components of standard deep learning pipelines, such as the batch size, data augmentation, and label smoothing, can achieve state-of-the-art performance without any such specialized class imbalance methods.
arXiv Detail & Related papers (2023-12-05T05:52:44Z) - CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep
Learning [55.733193075728096]
Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance.
Sample re-weighting methods are popularly used to alleviate this data bias issue.
We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
arXiv Detail & Related papers (2022-02-11T13:49:51Z) - FORML: Learning to Reweight Data for Fairness [2.105564340986074]
We introduce Fairness Optimized Reweighting via Meta-Learning (FORML)
FORML balances fairness constraints and accuracy by jointly optimizing training sample weights and a neural network's parameters.
We show that FORML improves equality of opportunity fairness criteria over existing state-of-the-art reweighting methods by approximately 1% on image classification tasks and by approximately 5% on a face prediction task.
arXiv Detail & Related papers (2022-02-03T17:36:07Z) - AutoLoss: Automated Loss Function Search in Recommendations [34.27873944762912]
We propose an AutoLoss framework that can automatically and adaptively search for the appropriate loss function from a set of candidates.
Unlike existing algorithms, the proposed controller can adaptively generate the loss probabilities for different data examples according to their varied convergence behaviors.
arXiv Detail & Related papers (2021-06-12T08:15:00Z) - Fed-Focal Loss for imbalanced data classification in Federated Learning [2.2172881631608456]
Federated Learning has a central server coordinating the training of a model on a network of devices.
One of the challenges is variable training performance when the dataset has a class imbalance.
We propose to address the class imbalance by reshaping cross-entropy loss such that it down-weights the loss assigned to well-classified examples along the lines of focal loss.
arXiv Detail & Related papers (2020-11-12T09:52:14Z) - Fairness Constraints in Semi-supervised Learning [56.48626493765908]
We develop a framework for fair semi-supervised learning, which is formulated as an optimization problem.
We theoretically analyze the source of discrimination in semi-supervised learning via bias, variance and noise decomposition.
Our method is able to achieve fair semi-supervised learning, and reach a better trade-off between accuracy and fairness than fair supervised learning.
arXiv Detail & Related papers (2020-09-14T04:25:59Z) - Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
We consider fairness in the integration component of data management.
We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
arXiv Detail & Related papers (2020-06-10T20:20:10Z) - Long-Tailed Recognition Using Class-Balanced Experts [128.73438243408393]
We propose an ensemble of class-balanced experts that combines the strength of diverse classifiers.
Our ensemble of class-balanced experts reaches results close to state-of-the-art and an extended ensemble establishes a new state-of-the-art on two benchmarks for long-tailed recognition.
arXiv Detail & Related papers (2020-04-07T20:57:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.