Towards Faithful Multimodal Concept Bottleneck Models

• URL: http://arxiv.org/abs/2603.13163v1
• Date: Fri, 13 Mar 2026 16:56:08 GMT
• Title: Towards Faithful Multimodal Concept Bottleneck Models
• Abstract summary: Concept Bottleneck Models (CBMs) are interpretable models that route predictions through a layer of human-interpretable concepts.<n>We introduce f-CBM, a multimodal CBM framework built on a vision-language backbone that jointly targets concept detection and leakage mitigation.<n>Experiments demonstrate that f-CBM achieves the best trade-off between task accuracy, concept detection, and leakage reduction.
• Score: 0.07329200485567826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Concept Bottleneck Models (CBMs) are interpretable models that route predictions through a layer of human-interpretable concepts. While widely studied in vision and, more recently, in NLP, CBMs remain largely unexplored in multimodal settings. For their explanations to be faithful, CBMs must satisfy two conditions: concepts must be properly detected, and concept representations must encode only their intended semantics, without smuggling extraneous task-relevant or inter-concept information into final predictions, a phenomenon known as leakage. Existing approaches treat concept detection and leakage mitigation as separate problems, and typically improve one at the expense of predictive accuracy. In this work, we introduce f-CBM, a faithful multimodal CBM framework built on a vision-language backbone that jointly targets both aspects through two complementary strategies: a differentiable leakage loss to mitigate leakage, and a Kolmogorov-Arnold Network prediction head that provides sufficient expressiveness to improve concept detection. Experiments demonstrate that f-CBM achieves the best trade-off between task accuracy, concept detection, and leakage reduction, while applying seamlessly to both image and text or text-only datasets, making it versatile across modalities.

Related papers

• Probabilistic Concept Graph Reasoning for Multimodal Misinformation Detection [50.08054676999716]
  We present Probabilistic Concept Graph Reasoning (PCGR), an interpretable and evolvable framework that reframes multimodal misinformation detection as structured and concept-based reasoning.<n>PCGR achieves state-of-the-art MMD accuracy and robustness to emerging manipulation types, outperforming prior methods in both coarse detection and fine-grained manipulation recognition.
  arXiv  Detail & Related papers  (2026-03-26T09:08:31Z)
• Rethinking Concept Bottleneck Models: From Pitfalls to Solutions [53.84388497227224]
  Concept Bottleneck Models (CBMs) ground predictions in human-understandable concepts.<n>CBM-Suite is a methodological framework to address these challenges.
  arXiv  Detail & Related papers  (2026-03-05T19:37:49Z)
• Concepts' Information Bottleneck Models [9.435622803973898]
  Concept Bottleneck Models (CBMs) aim to deliver interpretable predictions by routing decisions through a human-understandable concept layer.<n>We introduce an explicit Information Bottleneck regularizer on the concept layer that penalizes $I(X;C)$ while preserving task-relevant information in $I(C;Y)$, encouraging minimal-sufficient concept representations.
  arXiv  Detail & Related papers  (2026-02-16T10:33:20Z)
• Differential Vector Erasure: Unified Training-Free Concept Erasure for Flow Matching Models [49.10620605347065]
  We propose Differential Vector Erasure (DVE), a training-free concept erasure method specifically designed for flow matching models.<n>Our key insight is that semantic concepts are implicitly encoded in the directional structure of the velocity field governing the generative flow.<n>During inference, DVE selectively removes concept-specific components by projecting the velocity field onto the differential direction, enabling precise concept suppression without affecting irrelevant semantics.
  arXiv  Detail & Related papers  (2026-02-01T08:05:45Z)
• Controllable Concept Bottleneck Models [55.03639763625018]
  Controllable Concept Bottleneck Models (CCBMs)<n>CCBMs support three granularities of model editing: concept-label-level, concept-level, and data-level.<n>CCBMs enjoy mathematically rigorous closed-form approximations derived from influence functions that obviate the need for retraining.
  arXiv  Detail & Related papers  (2026-01-01T19:30:06Z)
• Towards more holistic interpretability: A lightweight disentangled Concept Bottleneck Model [5.700536552863068]
  Concept Bottleneck Models (CBMs) enhance interpretability by predicting human-understandable concepts as intermediate representations.<n>We propose a lightweight Disentangled Concept Bottleneck Model (LDCBM) that automatically groups visual features into semantically meaningful components.<n> Experiments on three diverse datasets demonstrate that LDCBM achieves higher concept and class accuracy, outperforming previous CBMs in both interpretability and classification performance.
  arXiv  Detail & Related papers  (2025-10-17T15:59:30Z)
• Interpretable Few-Shot Image Classification via Prototypical Concept-Guided Mixture of LoRA Experts [79.18608192761512]
  Self-Explainable Models (SEMs) rely on Prototypical Concept Learning (PCL) to enable their visual recognition processes more interpretable.<n>We propose a Few-Shot Prototypical Concept Classification framework that mitigates two key challenges under low-data regimes: parametric imbalance and representation misalignment.<n>Our approach consistently outperforms existing SEMs by a notable margin, with 4.2%-8.7% relative gains in 5-way 5-shot classification.
  arXiv  Detail & Related papers  (2025-06-05T06:39:43Z)
• Continuous Evolution Pool: Taming Recurring Concept Drift in Online Time Series Forecasting [58.448663215248565]
  Continuous Evolution Pool (CEP) is a pooling mechanism that stores different instances of forecasters for different concepts.<n>CEP effectively retains the knowledge of different concepts.<n>In the scenario of online forecasting with recurring concepts, CEP significantly enhances the prediction results.
  arXiv  Detail & Related papers  (2025-05-28T03:27:49Z)
• MulCPred: Learning Multi-modal Concepts for Explainable Pedestrian Action Prediction [57.483718822429346]
  MulCPred is proposed that explains its predictions based on multi-modal concepts represented by training samples. MulCPred is evaluated on multiple datasets and tasks.
  arXiv  Detail & Related papers  (2024-09-14T14:15:28Z)
• Eliminating Information Leakage in Hard Concept Bottleneck Models with Supervised, Hierarchical Concept Learning [17.982131928413096]
  Concept Bottleneck Models (CBMs) aim to deliver interpretable and interventionable predictions by bridging features and labels with human-understandable concepts. CBMs suffer from information leakage, where unintended information beyond the concepts are leaked to the subsequent label prediction. This paper proposes a new paradigm of CBMs, namely SupCBM, which achieves label predication via predicted concepts and a deliberately-designed intervention matrix.
  arXiv  Detail & Related papers  (2024-02-03T03:50:58Z)
• Auxiliary Losses for Learning Generalizable Concept-based Models [5.4066453042367435]
  Concept Bottleneck Models (CBMs) have gained popularity since their introduction. CBMs essentially limit the latent space of a model to human-understandable high-level concepts. We propose cooperative-Concept Bottleneck Model (coop-CBM) to overcome the performance trade-off.
  arXiv  Detail & Related papers  (2023-11-18T15:50:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.