Model-Based Approach for Measuring the Fairness in ASR

• URL: http://arxiv.org/abs/2109.09061v1
• Date: Sun, 19 Sep 2021 05:24:01 GMT
• Title: Model-Based Approach for Measuring the Fairness in ASR
• Authors: Zhe Liu, Irina-Elena Veliche, Fuchun Peng
• Abstract summary: We introduce mixed-effects Poisson regression to better measure and interpret any WER difference among subgroups of interest. We demonstrate the validity of proposed model-based approach on both synthetic and real-world speech data.
• Score: 11.076999352942954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The issue of fairness arises when the automatic speech recognition (ASR) systems do not perform equally well for all subgroups of the population. In any fairness measurement studies for ASR, the open questions of how to control the nuisance factors, how to handle unobserved heterogeneity across speakers, and how to trace the source of any word error rate (WER) gap among different subgroups are especially important - if not appropriately accounted for, incorrect conclusions will be drawn. In this paper, we introduce mixed-effects Poisson regression to better measure and interpret any WER difference among subgroups of interest. Particularly, the presented method can effectively address the three problems raised above and is very flexible to use in practical disparity analyses. We demonstrate the validity of proposed model-based approach on both synthetic and real-world speech data.

Related papers

• Uncovering Fairness through Data Complexity as an Early Indicator [0.7783262415147651]
  There is no study on how disparities in classification complexity between privileged and unprivileged groups could influence the fairness of solutions. In this work, we focus on synthetic datasets designed to capture a variety of biases ranging from historical bias to measurement and representational bias. We then apply association rule mining to identify patterns that link disproportionate complexity differences between groups with fairness-related outcomes.
  arXiv  Detail & Related papers  (2025-04-08T11:28:40Z)
• Unifying Causal Representation Learning with the Invariance Principle [21.375611599649716]
  Causal representation learning aims at recovering latent causal variables from high-dimensional observations. Our main contribution is to show that many existing causal representation learning approaches methodologically align the representation to known data symmetries.
  arXiv  Detail & Related papers  (2024-09-04T14:51:36Z)
• Reconciling Heterogeneous Effects in Causal Inference [44.99833362998488]
  We apply the Reconcile algorithm for model multiplicity in machine learning to reconcile heterogeneous effects in causal inference. Our results have tangible implications for ensuring fair outcomes in high-stakes such as healthcare, insurance, and housing.
  arXiv  Detail & Related papers  (2024-06-05T18:43:46Z)
• DINER: Debiasing Aspect-based Sentiment Analysis with Multi-variable Causal Inference [21.929902181609936]
  We propose a novel framework based on multi-variable causal inference for debiasing ABSA. For the review branch, the bias is modeled as indirect confounding from context, where backdoor adjustment intervention is employed for debiasing. For the aspect branch, the bias is described as a direct correlation with labels, where counterfactual reasoning is adopted for debiasing.
  arXiv  Detail & Related papers  (2024-03-02T10:38:31Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• Fairness Improves Learning from Noisily Labeled Long-Tailed Data [119.0612617460727]
  Long-tailed and noisily labeled data frequently appear in real-world applications and impose significant challenges for learning. We introduce the Fairness Regularizer (FR), inspired by regularizing the performance gap between any two sub-populations. We show that the introduced fairness regularizer improves the performances of sub-populations on the tail and the overall learning performance.
  arXiv  Detail & Related papers  (2023-03-22T03:46:51Z)
• Fairness via Adversarial Attribute Neighbourhood Robust Learning [49.93775302674591]
  We propose a principled underlineRobust underlineAdversarial underlineAttribute underlineNeighbourhood (RAAN) loss to debias the classification head.
  arXiv  Detail & Related papers  (2022-10-12T23:39:28Z)
• Beyond IID: data-driven decision-making in heterogeneous environments [8.714718004930363]
  We study a data-driven decision-making framework in which historical samples are generated from unknown and different distributions. This work aims at analyzing the performance of central data-driven policies but also near-optimal ones.
  arXiv  Detail & Related papers  (2022-06-20T08:43:43Z)
• Addressing Fairness, Bias and Class Imbalance in Machine Learning: the FBI-loss [11.291571222801027]
  We propose a unified loss correction to address issues related to Fairness, Biases and Imbalances (FBI-loss) The correction capabilities of the proposed approach are assessed on three real-world benchmarks.
  arXiv  Detail & Related papers  (2021-05-13T15:01:14Z)
• Can Active Learning Preemptively Mitigate Fairness Issues? [66.84854430781097]
  dataset bias is one of the prevailing causes of unfairness in machine learning. We study whether models trained with uncertainty-based ALs are fairer in their decisions with respect to a protected class. We also explore the interaction of algorithmic fairness methods such as gradient reversal (GRAD) and BALD.
  arXiv  Detail & Related papers  (2021-04-14T14:20:22Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• Learning Overlapping Representations for the Estimation of Individualized Treatment Effects [97.42686600929211]
  Estimating the likely outcome of alternatives from observational data is a challenging problem. We show that algorithms that learn domain-invariant representations of inputs are often inappropriate. We develop a deep kernel regression algorithm and posterior regularization framework that substantially outperforms the state-of-the-art on a variety of benchmarks data sets.
  arXiv  Detail & Related papers  (2020-01-14T12:56:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.