DCBM: Data-Efficient Visual Concept Bottleneck Models

• URL: http://arxiv.org/abs/2412.11576v2
• Date: Tue, 04 Feb 2025 15:41:34 GMT
• Title: DCBM: Data-Efficient Visual Concept Bottleneck Models
• Authors: Katharina Prasse, Patrick Knab, Sascha Marton, Christian Bartelt, Margret Keuper,
• Abstract summary: Concept Bottleneck Models (CBMs) enhance interpretability of neural networks by basing predictions on human-understandable concepts. We propose Data-efficient CBMs, which reduce the need for large sample sizes during concept generation while preserving interpretability.
• Score: 13.36057999450821
• License:
• Abstract: Concept Bottleneck Models (CBMs) enhance the interpretability of neural networks by basing predictions on human-understandable concepts. However, current CBMs typically rely on concept sets extracted from large language models or extensive image corpora, limiting their effectiveness in data-sparse scenarios. We propose Data-efficient CBMs (DCBMs), which reduce the need for large sample sizes during concept generation while preserving interpretability. DCBMs define concepts as image regions detected by segmentation or detection foundation models, allowing each image to generate multiple concepts across different granularities. This removes reliance on textual descriptions and large-scale pre-training, making DCBMs applicable for fine-grained classification and out-of-distribution tasks. Attribution analysis using Grad-CAM demonstrates that DCBMs deliver visual concepts that can be localized in test images. By leveraging dataset-specific concepts instead of predefined ones, DCBMs enhance adaptability to new domains.

Related papers

• V2C-CBM: Building Concept Bottlenecks with Vision-to-Concept Tokenizer [19.177297480709512]
  Concept Bottleneck Models (CBMs) offer inherent interpretability by translating images into human-comprehensible concepts. Recent approaches have leveraged the knowledge of large language models to construct concept bottlenecks. In this study, we investigate to avoid these issues by constructing CBMs directly from multimodal models.
  arXiv  Detail & Related papers  (2025-01-09T05:12:38Z)
• How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization? [91.49559116493414]
  We propose a novel Concept-Incremental text-to-image Diffusion Model (CIDM) It can resolve catastrophic forgetting and concept neglect to learn new customization tasks in a concept-incremental manner. Experiments validate that our CIDM surpasses existing custom diffusion models.
  arXiv  Detail & Related papers  (2024-10-23T06:47:29Z)
• Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery [52.498055901649025]
  Concept Bottleneck Models (CBMs) have been proposed to address the 'black-box' problem of deep neural networks. We propose a novel CBM approach -- called Discover-then-Name-CBM (DN-CBM) -- that inverts the typical paradigm. Our concept extraction strategy is efficient, since it is agnostic to the downstream task, and uses concepts already known to the model.
  arXiv  Detail & Related papers  (2024-07-19T17:50:11Z)
• Improving Intervention Efficacy via Concept Realignment in Concept Bottleneck Models [57.86303579812877]
  Concept Bottleneck Models (CBMs) ground image classification on human-understandable concepts to allow for interpretable model decisions. Existing approaches often require numerous human interventions per image to achieve strong performances. We introduce a trainable concept realignment intervention module, which leverages concept relations to realign concept assignments post-intervention.
  arXiv  Detail & Related papers  (2024-05-02T17:59:01Z)
• Incremental Residual Concept Bottleneck Models [29.388549499546556]
  Concept Bottleneck Models (CBMs) map the black-box visual representations extracted by deep neural networks onto a set of interpretable concepts. We propose the Incremental Residual Concept Bottleneck Model (Res-CBM) to address the challenge of concept completeness. Our approach can be applied to any user-defined concept bank, as a post-hoc processing method to enhance the performance of any CBMs.
  arXiv  Detail & Related papers  (2024-04-13T12:02:19Z)
• Visual Concept-driven Image Generation with Text-to-Image Diffusion Model [65.96212844602866]
  Text-to-image (TTI) models have demonstrated impressive results in generating high-resolution images of complex scenes. Recent approaches have extended these methods with personalization techniques that allow them to integrate user-illustrated concepts. However, the ability to generate images with multiple interacting concepts, such as human subjects, as well as concepts that may be entangled in one, or across multiple, image illustrations remains illusive. We propose a concept-driven TTI personalization framework that addresses these core challenges.
  arXiv  Detail & Related papers  (2024-02-18T07:28:37Z)
• Auxiliary Losses for Learning Generalizable Concept-based Models [5.4066453042367435]
  Concept Bottleneck Models (CBMs) have gained popularity since their introduction. CBMs essentially limit the latent space of a model to human-understandable high-level concepts. We propose cooperative-Concept Bottleneck Model (coop-CBM) to overcome the performance trade-off.
  arXiv  Detail & Related papers  (2023-11-18T15:50:07Z)
• I saw, I conceived, I concluded: Progressive Concepts as Bottlenecks [2.9398911304923447]
  Concept bottleneck models (CBMs) provide explainability and intervention during inference by correcting predicted, intermediate concepts. This makes CBMs attractive for high-stakes decision-making. We take the quality assessment of fetal ultrasound scans as a real-life use case for CBM decision support in healthcare.
  arXiv  Detail & Related papers  (2022-11-19T09:31:19Z)
• Post-hoc Concept Bottleneck Models [11.358495577593441]
  Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable concepts and use the concepts to make predictions. CBMs are restrictive in practice as they require concept labels in the training data to learn the bottleneck and do not leverage strong pretrained models. We show that we can turn any neural network into a PCBM without sacrificing model performance while still retaining interpretability benefits.
  arXiv  Detail & Related papers  (2022-05-31T00:29:26Z)
• Modeling Temporal Concept Receptive Field Dynamically for Untrimmed Video Analysis [105.06166692486674]
  We study temporal concept receptive field of concept-based event representation. We introduce temporal dynamic convolution (TDC) to give stronger flexibility to concept-based event analytics. Different coefficients can generate appropriate and accurate temporal concept receptive field size according to input videos.
  arXiv  Detail & Related papers  (2021-11-23T04:59:48Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.