Measuring directional bias amplification in image captions using predictability

• URL: http://arxiv.org/abs/2503.07878v2
• Date: Wed, 12 Mar 2025 02:47:54 GMT
• Title: Measuring directional bias amplification in image captions using predictability
• Authors: Rahul Nair, Bhanu Tokas, Neel Shah, Hannah Kerner,
• Abstract summary: We propose Directional Predictability Amplification in Captioning (DPAC) to measure bias amplification in ML datasets.<n>DPAC measures directional bias amplification in captions, provides a better estimate of dataset bias, and is less sensitive to attacker models.<n>Our experiments on the COCO captioning dataset show how DPAC is the most reliable metric to measure bias amplification in captions.
• Score: 13.041091740013808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When we train models on biased ML datasets, they not only learn these biases but can inflate them at test time - a phenomenon called bias amplification. To measure bias amplification in ML datasets, many co-occurrence-based metrics have been proposed. Co-occurrence-based metrics are effective in measuring bias amplification in simple problems like image classification. However, these metrics are ineffective for complex problems like image captioning as they cannot capture the semantics of a caption. To measure bias amplification in captions, prior work introduced a predictability-based metric called Leakage in Captioning (LIC). While LIC captures the semantics and context of captions, it has limitations. LIC cannot identify the direction in which bias is amplified, poorly estimates dataset bias due to a weak vocabulary substitution strategy, and is highly sensitive to attacker models (a hyperparameter in predictability-based metrics). To overcome these issues, we propose Directional Predictability Amplification in Captioning (DPAC). DPAC measures directional bias amplification in captions, provides a better estimate of dataset bias using an improved substitution strategy, and is less sensitive to attacker models. Our experiments on the COCO captioning dataset show how DPAC is the most reliable metric to measure bias amplification in captions.

Related papers

• Making Bias Amplification in Balanced Datasets Directional and Interpretable [13.38327450225136]
  We propose a new predictability-based metric called directional predictability amplification (DPA)<n>DPA measures directional bias amplification, even for balanced datasets.<n>Our experiments show that DPA is an effective metric for measuring directional bias amplification.
  arXiv  Detail & Related papers  (2024-12-15T05:32:54Z)
• Revisiting the Dataset Bias Problem from a Statistical Perspective [72.94990819287551]
  We study the "dataset bias" problem from a statistical standpoint. We identify the main cause of the problem as the strong correlation between a class attribute u and a non-class attribute b. We propose to mitigate dataset bias via either weighting the objective of each sample n by frac1p(u_n|b_n) or sampling that sample with a weight proportional to frac1p(u_n|b_n).
  arXiv  Detail & Related papers  (2024-02-05T22:58:06Z)
• Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Sets [52.77024349608834]
  Vision-language models can perpetuate and amplify societal biases learned during pre-training on uncurated image-text pairs from the internet. COCO Captions is the most commonly used dataset for evaluating bias between background context and the gender of people in-situ. We propose a novel dataset debiasing pipeline to augment the COCO dataset with synthetic, gender-balanced contrast sets.
  arXiv  Detail & Related papers  (2023-05-24T17:59:18Z)
• ImageCaptioner$^2$: Image Captioner for Image Captioning Bias Amplification Assessment [30.71835197717301]
  We introduce a new bias assessment metric, dubbed $ImageCaptioner2$, for image captioning. Instead of measuring the absolute bias in the model or the data, $ImageCaptioner2$ pay more attention to the bias introduced by the model w.r.t the data bias. In addition, we design a formulation for measuring the bias of generated captions as prompt-based image captioning.
  arXiv  Detail & Related papers  (2023-04-10T21:40:46Z)
• Feature-Level Debiased Natural Language Understanding [86.8751772146264]
  Existing natural language understanding (NLU) models often rely on dataset biases to achieve high performance on specific datasets. We propose debiasing contrastive learning (DCT) to mitigate biased latent features and neglect the dynamic nature of bias. DCT outperforms state-of-the-art baselines on out-of-distribution datasets while maintaining in-distribution performance.
  arXiv  Detail & Related papers  (2022-12-11T06:16:14Z)
• ADEPT: A DEbiasing PrompT Framework [49.582497203415855]
  Finetuning is an applicable approach for debiasing contextualized word embeddings. discrete prompts with semantic meanings have shown to be effective in debiasing tasks. We propose ADEPT, a method to debias PLMs using prompt tuning while maintaining the delicate balance between removing biases and ensuring representation ability.
  arXiv  Detail & Related papers  (2022-11-10T08:41:40Z)
• Men Also Do Laundry: Multi-Attribute Bias Amplification [2.492300648514129]
  Computer vision systems are not only reproducing but amplifying harmful social biases. We propose a new metric: Multi-Attribute Bias Amplification. We validate our proposed metric through an analysis of gender bias amplification on the COCO and imSitu datasets.
  arXiv  Detail & Related papers  (2022-10-21T12:50:15Z)
• Quantifying Societal Bias Amplification in Image Captioning [24.075869811508404]
  We argue that, for image captioning, it is not enough to focus on the correct prediction of the protected attribute, and the whole context should be taken into account. We conduct extensive evaluation on traditional and state-of-the-art image captioning models, and surprisingly find that, by only focusing on the protected attribute prediction, bias mitigation models are unexpectedly amplifying bias.
  arXiv  Detail & Related papers  (2022-03-29T09:42:11Z)
• A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning [55.96577490779591]
  Vision-language models can encode societal biases and stereotypes. There are challenges to measuring and mitigating these multimodal harms. We investigate bias measures and apply ranking metrics for image-text representations.
  arXiv  Detail & Related papers  (2022-03-22T17:59:04Z)
• Information-Theoretic Bias Reduction via Causal View of Spurious Correlation [71.9123886505321]
  We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation. We present a novel debiasing framework against the algorithmic bias, which incorporates a bias regularization loss. The proposed bias measurement and debiasing approaches are validated in diverse realistic scenarios.
  arXiv  Detail & Related papers  (2022-01-10T01:19:31Z)
• Directional Bias Amplification [21.482317675176443]
  Bias amplification is the tendency of models to amplify the biases present in the data they are trained on. A metric for measuring bias amplification was introduced in the seminal work by Zhao et al. We introduce and analyze a new, decoupled metric for measuring bias amplification, $textBiasAmp_rightarrow$ (Directional Bias Amplification)
  arXiv  Detail & Related papers  (2021-02-24T22:54:21Z)
• Improving Robustness by Augmenting Training Sentences with Predicate-Argument Structures [62.562760228942054]
  Existing approaches to improve robustness against dataset biases mostly focus on changing the training objective. We propose to augment the input sentences in the training data with their corresponding predicate-argument structures. We show that without targeting a specific bias, our sentence augmentation improves the robustness of transformer models against multiple biases.
  arXiv  Detail & Related papers  (2020-10-23T16:22:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.