Exploring Text Specific and Blackbox Fairness Algorithms in Multimodal Clinical NLP

• URL: http://arxiv.org/abs/2011.09625v2
• Date: Thu, 10 Jun 2021 15:54:39 GMT
• Title: Exploring Text Specific and Blackbox Fairness Algorithms in Multimodal Clinical NLP
• Authors: John Chen, Ian Berlot-Attwell, Safwan Hossain, Xindi Wang and Frank Rudzicz
• Abstract summary: We investigate a modality-agnostic fairness algorithm - equalized odds post processing - and compare it to a text-specific fairness algorithm: debiased clinical word embeddings. Despite the fact that debiased word embeddings do not explicitly address equalized odds of protected groups, we show that a text-specific approach to fairness may simultaneously achieve a good balance of performance and classical notions of fairness.
• Score: 19.936882021907266
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Clinical machine learning is increasingly multimodal, collected in both structured tabular formats and unstructured forms such as freetext. We propose a novel task of exploring fairness on a multimodal clinical dataset, adopting equalized odds for the downstream medical prediction tasks. To this end, we investigate a modality-agnostic fairness algorithm - equalized odds post processing - and compare it to a text-specific fairness algorithm: debiased clinical word embeddings. Despite the fact that debiased word embeddings do not explicitly address equalized odds of protected groups, we show that a text-specific approach to fairness may simultaneously achieve a good balance of performance and classical notions of fairness. We hope that our paper inspires future contributions at the critical intersection of clinical NLP and fairness. The full source code is available here: https://github.com/johntiger1/multimodal_fairness

Related papers

• Post-processing fairness with minimal changes [5.927938174149359]
  We introduce a novel post-processing algorithm that is both model-agnostic and does not require the sensitive attribute at test time. Our algorithm is explicitly designed to enforce minimal changes between biased and debiased predictions.
  arXiv  Detail & Related papers  (2024-08-27T14:26:56Z)
• Fair Text Classification with Wasserstein Independence [4.887319013701134]
  Group fairness is a central research topic in text classification, where reaching fair treatment between sensitive groups remains an open challenge. This paper presents a novel method for mitigating biases in neural text classification, agnostic to the model architecture.
  arXiv  Detail & Related papers  (2023-11-21T15:51:06Z)
• FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods [84.1077756698332]
  This paper introduces the Fair Fairness Benchmark (textsfFFB), a benchmarking framework for in-processing group fairness methods. We provide a comprehensive analysis of state-of-the-art methods to ensure different notions of group fairness.
  arXiv  Detail & Related papers  (2023-06-15T19:51:28Z)
• A Multi-View Joint Learning Framework for Embedding Clinical Codes and Text Using Graph Neural Networks [23.06795121693656]
  We propose a framework that learns from codes and text to combine the availability and forward-looking nature of text and better performance of ICD codes. Our approach uses a Graph Neural Network (GNN) to process ICD codes, and Bi-LSTM to process text. In experiments using planned surgical procedure text, our model outperforms BERT models fine-tuned to clinical data.
  arXiv  Detail & Related papers  (2023-01-27T09:19:03Z)
• FaiREE: Fair Classification with Finite-Sample and Distribution-Free Guarantee [40.10641140860374]
  FaiREE is a fair classification algorithm that can satisfy group fairness constraints with finite-sample and distribution-free theoretical guarantees. FaiREE is shown to have favorable performance over state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2022-11-28T05:16:20Z)
• Practical Approaches for Fair Learning with Multitype and Multivariate Sensitive Attributes [70.6326967720747]
  It is important to guarantee that machine learning algorithms deployed in the real world do not result in unfairness or unintended social consequences. We introduce FairCOCCO, a fairness measure built on cross-covariance operators on reproducing kernel Hilbert Spaces. We empirically demonstrate consistent improvements against state-of-the-art techniques in balancing predictive power and fairness on real-world datasets.
  arXiv  Detail & Related papers  (2022-11-11T11:28:46Z)
• Textual Entailment Recognition with Semantic Features from Empirical Text Representation [60.31047947815282]
  A text entails a hypothesis if and only if the true value of the hypothesis follows the text. In this paper, we propose a novel approach to identifying the textual entailment relationship between text and hypothesis. We employ an element-wise Manhattan distance vector-based feature that can identify the semantic entailment relationship between the text-hypothesis pair.
  arXiv  Detail & Related papers  (2022-10-18T10:03:51Z)
• Fair Wrapping for Black-box Predictions [105.10203274098862]
  We learn a wrapper function which we define as an alpha-tree, which modifies the prediction. We show that our modification has appealing properties in terms of composition ofalpha-trees, generalization, interpretability, and KL divergence between modified and original predictions.
  arXiv  Detail & Related papers  (2022-01-31T01:02:39Z)
• Intrinsic Probing through Dimension Selection [69.52439198455438]
  Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of tasks. Such high performance should not be possible unless some form of linguistic structure inheres in these representations, and a wealth of research has sprung up on probing for it. In this paper, we draw a distinction between intrinsic probing, which examines how linguistic information is structured within a representation, and the extrinsic probing popular in prior work, which only argues for the presence of such information by showing that it can be successfully extracted.
  arXiv  Detail & Related papers  (2020-10-06T15:21:08Z)
• Causal Feature Selection for Algorithmic Fairness [61.767399505764736]
  We consider fairness in the integration component of data management. We propose an approach to identify a sub-collection of features that ensure the fairness of the dataset.
  arXiv  Detail & Related papers  (2020-06-10T20:20:10Z)
• Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings [16.136832979324467]
  We pretrain deep embedding models (BERT) on medical notes from the MIMIC-III hospital dataset. We identify dangerous latent relationships that are captured by the contextual word embeddings. We evaluate performance gaps across different definitions of fairness on over 50 downstream clinical prediction tasks.
  arXiv  Detail & Related papers  (2020-03-11T23:21:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.