Fairness meets Cross-Domain Learning: a new perspective on Models and Metrics

• URL: http://arxiv.org/abs/2303.14411v1
• Date: Sat, 25 Mar 2023 09:34:05 GMT
• Title: Fairness meets Cross-Domain Learning: a new perspective on Models and Metrics
• Authors: Leonardo Iurada, Silvia Bucci, Timothy M. Hospedales, Tatiana Tommasi
• Abstract summary: We study the relationship between cross-domain learning (CD) and model fairness. We introduce a benchmark on face and medical images spanning several demographic groups as well as classification and localization tasks. Our study covers 14 CD approaches alongside three state-of-the-art fairness algorithms and shows how the former can outperform the latter.
• Score: 80.07271410743806
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning-based recognition systems are deployed at scale for several real-world applications that inevitably involve our social life. Although being of great support when making complex decisions, they might capture spurious data correlations and leverage sensitive attributes (e.g. age, gender, ethnicity). How to factor out this information while keeping a high prediction performance is a task with still several open questions, many of which are shared with those of the domain adaptation and generalization literature which focuses on avoiding visual domain biases. In this work, we propose an in-depth study of the relationship between cross-domain learning (CD) and model fairness by introducing a benchmark on face and medical images spanning several demographic groups as well as classification and localization tasks. After having highlighted the limits of the current evaluation metrics, we introduce a new Harmonic Fairness (HF) score to assess jointly how fair and accurate every model is with respect to a reference baseline. Our study covers 14 CD approaches alongside three state-of-the-art fairness algorithms and shows how the former can outperform the latter. Overall, our work paves the way for a more systematic analysis of fairness problems in computer vision. Code available at: https://github.com/iurada/fairness_crossdomain

Related papers

• A Benchmark for Fairness-Aware Graph Learning [58.515305543487386]
  We present an extensive benchmark on ten representative fairness-aware graph learning methods. Our in-depth analysis reveals key insights into the strengths and limitations of existing methods.
  arXiv  Detail & Related papers  (2024-07-16T18:43:43Z)
• Bridging the Gap: Protocol Towards Fair and Consistent Affect Analysis [24.737468736951374]
  The increasing integration of machine learning algorithms in daily life underscores the critical need for fairness and equity in their deployment. Existing databases and methodologies lack uniformity, leading to biased evaluations. This work addresses these issues by analyzing six affective databases, annotating demographic attributes, and proposing a common protocol for database partitioning.
  arXiv  Detail & Related papers  (2024-05-10T22:40:01Z)
• Evaluating the Fairness of Discriminative Foundation Models in Computer Vision [51.176061115977774]
  We propose a novel taxonomy for bias evaluation of discriminative foundation models, such as Contrastive Language-Pretraining (CLIP) We then systematically evaluate existing methods for mitigating bias in these models with respect to our taxonomy. Specifically, we evaluate OpenAI's CLIP and OpenCLIP models for key applications, such as zero-shot classification, image retrieval and image captioning.
  arXiv  Detail & Related papers  (2023-10-18T10:32:39Z)
• Towards Reliable Assessments of Demographic Disparities in Multi-Label Image Classifiers [11.973749734226852]
  We consider multi-label image classification and, specifically, object categorization tasks. Design choices and trade-offs for measurement involve more nuance than discussed in prior computer vision literature. We identify several design choices that look merely like implementation details but significantly impact the conclusions of assessments.
  arXiv  Detail & Related papers  (2023-02-16T20:34:54Z)
• A Survey on Fairness for Machine Learning on Graphs [2.3326951882644553]
  This survey is the first one dedicated to fairness for relational data. It aims to present a comprehensive review of state-of-the-art techniques in fairness on graph mining.
  arXiv  Detail & Related papers  (2022-05-11T10:40:56Z)
• Fairness Indicators for Systematic Assessments of Visual Feature Extractors [21.141633753573764]
  We propose three fairness indicators, which aim at quantifying harms and biases of visual systems. Our indicators use existing publicly available datasets collected for fairness evaluations. These indicators are not intended to be a substitute for a thorough analysis of the broader impact of the new computer vision technologies.
  arXiv  Detail & Related papers  (2022-02-15T17:45:33Z)
• MultiFair: Multi-Group Fairness in Machine Learning [52.24956510371455]
  We study multi-group fairness in machine learning (MultiFair) We propose a generic end-to-end algorithmic framework to solve it. Our proposed framework is generalizable to many different settings.
  arXiv  Detail & Related papers  (2021-05-24T02:30:22Z)
• Through the Data Management Lens: Experimental Analysis and Evaluation of Fair Classification [75.49600684537117]
  Data management research is showing an increasing presence and interest in topics related to data and algorithmic fairness. We contribute a broad analysis of 13 fair classification approaches and additional variants, over their correctness, fairness, efficiency, scalability, and stability. Our analysis highlights novel insights on the impact of different metrics and high-level approach characteristics on different aspects of performance.
  arXiv  Detail & Related papers  (2021-01-18T22:55:40Z)
• Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning [85.6386289476598]
  We develop a novel adversarial graph representation adaptation (AGRA) framework for cross-domain holistic-local feature co-adaptation. We conduct extensive and fair evaluations on several popular benchmarks and show that the proposed AGRA framework outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-03T15:00:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.