Mitigating Bias in Concept Bottleneck Models for Fair and Interpretable Image Classification

• URL: http://arxiv.org/abs/2603.05899v1
• Date: Fri, 06 Mar 2026 04:37:23 GMT
• Title: Mitigating Bias in Concept Bottleneck Models for Fair and Interpretable Image Classification
• Abstract summary: Concept bottleneck models (CBMs) map images to high-level, human-interpretable concepts before making predictions.<n>CBMs have been known to leak information unrelated to concept semantics.<n>We propose three bias mitigation techniques to improve fairness in CBMs.
• Score: 0.5219568203653523
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ensuring fairness in image classification prevents models from perpetuating and amplifying bias. Concept bottleneck models (CBMs) map images to high-level, human-interpretable concepts before making predictions via a sparse, one-layer classifier. This structure enhances interpretability and, in theory, supports fairness by masking sensitive attribute proxies such as facial features. However, CBM concepts have been known to leak information unrelated to concept semantics and early results reveal only marginal reductions in gender bias on datasets like ImSitu. We propose three bias mitigation techniques to improve fairness in CBMs: 1. Decreasing information leakage using a top-k concept filter, 2. Removing biased concepts, and 3. Adversarial debiasing. Our results outperform prior work in terms of fairness-performance tradeoffs, indicating that our debiased CBM provides a significant step towards fair and interpretable image classification.

Related papers

• Rethinking Concept Bottleneck Models: From Pitfalls to Solutions [53.84388497227224]
  Concept Bottleneck Models (CBMs) ground predictions in human-understandable concepts.<n>CBM-Suite is a methodological framework to address these challenges.
  arXiv  Detail & Related papers  (2026-03-05T19:37:49Z)
• Towards more holistic interpretability: A lightweight disentangled Concept Bottleneck Model [5.700536552863068]
  Concept Bottleneck Models (CBMs) enhance interpretability by predicting human-understandable concepts as intermediate representations.<n>We propose a lightweight Disentangled Concept Bottleneck Model (LDCBM) that automatically groups visual features into semantically meaningful components.<n> Experiments on three diverse datasets demonstrate that LDCBM achieves higher concept and class accuracy, outperforming previous CBMs in both interpretability and classification performance.
  arXiv  Detail & Related papers  (2025-10-17T15:59:30Z)
• Uncertainty-Aware Concept Bottleneck Models with Enhanced Interpretability [2.624902795082451]
  Concept Bottleneck Models (CBMs) first embed images into a set of human-understandable concepts.<n>CBMs offer a semantically meaningful and interpretable classification pipeline.<n>CBMs often sacrifice predictive performance compared to end-to-end convolutional neural networks.
  arXiv  Detail & Related papers  (2025-10-01T11:11:18Z)
• RoboView-Bias: Benchmarking Visual Bias in Embodied Agents for Robotic Manipulation [67.38036090822982]
  We propose RoboView-Bias, the first benchmark specifically designed to quantify visual bias in robotic manipulation.<n>We create 2,127 task instances that enable robust measurement of biases induced by individual visual factors and their interactions.<n>Our results highlight that systematic analysis of visual bias is a prerequisite for developing safe and reliable general-purpose embodied agents.
  arXiv  Detail & Related papers  (2025-09-26T13:53:25Z)
• BiasMap: Leveraging Cross-Attentions to Discover and Mitigate Hidden Social Biases in Text-to-Image Generation [14.110668963732273]
  BiasMap is a model-agnostic framework for uncovering latent concept-level representational biases.<n>Our findings show that existing fairness interventions may reduce the output distributional gap but often fail to disentangle concept-level coupling.
  arXiv  Detail & Related papers  (2025-09-16T19:52:12Z)
• Interpretable Few-Shot Image Classification via Prototypical Concept-Guided Mixture of LoRA Experts [79.18608192761512]
  Self-Explainable Models (SEMs) rely on Prototypical Concept Learning (PCL) to enable their visual recognition processes more interpretable.<n>We propose a Few-Shot Prototypical Concept Classification framework that mitigates two key challenges under low-data regimes: parametric imbalance and representation misalignment.<n>Our approach consistently outperforms existing SEMs by a notable margin, with 4.2%-8.7% relative gains in 5-way 5-shot classification.
  arXiv  Detail & Related papers  (2025-06-05T06:39:43Z)
• Discovering Fine-Grained Visual-Concept Relations by Disentangled Optimal Transport Concept Bottleneck Models [16.617257464664572]
  Concept Bottleneck Models (CBMs) try to make the decision-making process transparent by exploring an intermediate concept space between the input image and the output prediction.<n>Existing CBMs just learn coarse-grained relations between the whole image and the concepts, less considering local image information.<n>This paper proposes a Disentangled Optimal Transport CBM framework to explore fine-grained visual-concept relations between local image patches and concepts.
  arXiv  Detail & Related papers  (2025-05-12T03:31:57Z)
• A Meaningful Perturbation Metric for Evaluating Explainability Methods [55.09730499143998]
  We introduce a novel approach, which harnesses image generation models to perform targeted perturbation.<n> Specifically, we focus on inpainting only the high-relevance pixels of an input image to modify the model's predictions while preserving image fidelity.<n>This is in contrast to existing approaches, which often produce out-of-distribution modifications, leading to unreliable results.
  arXiv  Detail & Related papers  (2025-04-09T11:46:41Z)
• Identifying and Mitigating Social Bias Knowledge in Language Models [52.52955281662332]
  We propose a novel debiasing approach, Fairness Stamp (FAST), which enables fine-grained calibration of individual social biases.<n>FAST surpasses state-of-the-art baselines with superior debiasing performance.<n>This highlights the potential of fine-grained debiasing strategies to achieve fairness in large language models.
  arXiv  Detail & Related papers  (2024-08-07T17:14:58Z)
• Classes Are Not Equal: An Empirical Study on Image Recognition Fairness [100.36114135663836]
  We experimentally demonstrate that classes are not equal and the fairness issue is prevalent for image classification models across various datasets. Our findings reveal that models tend to exhibit greater prediction biases for classes that are more challenging to recognize. Data augmentation and representation learning algorithms improve overall performance by promoting fairness to some degree in image classification.
  arXiv  Detail & Related papers  (2024-02-28T07:54:50Z)
• Eliminating Information Leakage in Hard Concept Bottleneck Models with Supervised, Hierarchical Concept Learning [17.982131928413096]
  Concept Bottleneck Models (CBMs) aim to deliver interpretable and interventionable predictions by bridging features and labels with human-understandable concepts. CBMs suffer from information leakage, where unintended information beyond the concepts are leaked to the subsequent label prediction. This paper proposes a new paradigm of CBMs, namely SupCBM, which achieves label predication via predicted concepts and a deliberately-designed intervention matrix.
  arXiv  Detail & Related papers  (2024-02-03T03:50:58Z)
• Explainable fetal ultrasound quality assessment with progressive concept bottleneck models [6.734637459963132]
  We propose a holistic and explainable method for fetal ultrasound quality assessment.<n>We introduce human-readable concepts" into the task and imitate the sequential expert decision-making process.<n> Experiments show that our model outperforms equivalent concept-free models on an in-house dataset.
  arXiv  Detail & Related papers  (2022-11-19T09:31:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.