When Interpretability Becomes a Liability: Adversarial Attacks on CBM Concept Layers

• URL: http://arxiv.org/abs/2605.25304v1
• Date: Mon, 25 May 2026 00:03:47 GMT
• Title: When Interpretability Becomes a Liability: Adversarial Attacks on CBM Concept Layers
• Abstract summary: Concept Bottleneck Models (CBMs) have emerged as a cornerstone approach for interpretable machine learning.<n>We present a comprehensive, systematic study of concept-level adversarial vulnerabilities in CBMs.<n>We introduce SPECTRA, a stability principled regularization defense.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Concept Bottleneck Models (CBMs) have emerged as a cornerstone approach for interpretable machine learning, providing human-understandable intermediate representations through explicit concept activations. However, this interpretability fundamentally introduces a critical, previously unexplored attack surface: the concept bottleneck layer itself. We present a comprehensive, systematic study of concept-level adversarial vulnerabilities in CBMs, revealing that targeted, minimal perturbations operating on input pixels can induce catastrophic misclassification by manipulating semantic representations. We develop a rigorous theoretical framework to quantify concept-space robustness, establishing novel metrics that expose the vulnerability landscape of these architectures. Our extensive analysis on the CUB-200-2011 dataset demonstrates that standard CBMs exhibit severe susceptibility to concept-level manipulation. To address this critical weakness, we introduce SPECTRA (Semantic Perturbation-based Concept Training for Robustness against Attacks), a principled stability regularization defense. SPECTRA effectively hardens the semantic representation space, increasing the minimal perturbation norm required for a successful attack from 0.46 to over 4,200, rendering targeted concept manipulation computationally prohibitive. Furthermore, SPECTRA preserves baseline classification accuracy to within 2.2%. By establishing concept-level attacks as a fundamentally distinct threat model, this work opens a new research frontier at the intersection of interpretable machine learning and adversarial robustness.

Related papers

• Weakly supervised concept Bottleneck Learning for Robust Two stage Object centric visual reasoning [3.6959117433332724]
  Two-stage neuro-symbolic architectures provide an elegant paradigm for visual problem solving.<n>We introduce the Dynamic Orthogonal Concept Bottleneck (D-OCB), an object-centric slot- VAE framework.
  arXiv  Detail & Related papers  (2026-08-23T20:39:49Z)
• Provably Secure Agent Guardrail [89.79561918065122]
  Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
  arXiv  Detail & Related papers  (2026-05-28T02:12:41Z)
• Hyperbolic Concept Bottleneck Models [17.962222104908648]
  Concept Bottleneck Models (CBMs) have become a popular approach to enable interpretability in neural networks by constraining inputs to a set of human-understandable concepts.<n>We propose Hyperbolic Concept Bottleneck Models (HypCBM), a post-hoc framework that grounds the bottleneck in this structure by reformulating concept activation as asymmetric geometric containment in hyperbolic space.<n>We show they encode a natural test-time activation signal: the margin of inclusion within a concept's entailment cone yields sparse, hierarchy-aware activations without any additional supervision or learned modules.
  arXiv  Detail & Related papers  (2026-05-07T15:41:22Z)
• Rethinking Concept Bottleneck Models: From Pitfalls to Solutions [53.84388497227224]
  Concept Bottleneck Models (CBMs) ground predictions in human-understandable concepts.<n>CBM-Suite is a methodological framework to address these challenges.
  arXiv  Detail & Related papers  (2026-03-05T19:37:49Z)
• Understanding Degradation with Vision Language Model [56.09241449206817]
  Understanding visual degradations is a critical yet challenging problem in computer vision.<n>We introduce DU-VLM, a multimodal chain-of-thought model trained with supervised fine-tuning and reinforcement learning.<n>We also introduce textbfDU-110k, a large-scale dataset comprising 110,000 clean-degraded pairs with grounded physical annotations.
  arXiv  Detail & Related papers  (2026-02-04T13:51:15Z)
• Rethinking Transferable Adversarial Attacks on Point Clouds from a Compact Subspace Perspective [55.919842734983156]
  CoSA is a transferable attack framework that operates within a shared low-dimensional semantic space.<n>CoSA consistently outperforms state-of-the-art transferable attacks.
  arXiv  Detail & Related papers  (2026-01-30T15:48:11Z)
• Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions [13.877511370053794]
  Concept Bottleneck Models (CBM) address some of these challenges by learning interpretable concepts from high-dimensional data.<n>We describe a framework that provides theoretical guarantees on the correctness of the learned concepts and on the number of required labels.<n>We evaluate our framework in synthetic and image benchmarks, showing that the learned concepts have less impurities and are often more accurate than other CBMs.
  arXiv  Detail & Related papers  (2025-02-10T15:01:56Z)
• Guarding the Gate: ConceptGuard Battles Concept-Level Backdoors in Concept Bottleneck Models [8.793955189563516]
  Concept Bottleneck Models (CBMs) enhance transparency by using high-level semantic concepts. CBMs are vulnerable to concept-level backdoor attacks, which inject hidden triggers into these concepts, leading to undetectable anomalous behavior. We introduce ConceptGuard, a novel defense framework specifically designed to protect CBMs from concept-level backdoor attacks.
  arXiv  Detail & Related papers  (2024-11-25T15:55:06Z)
• CAT: Concept-level backdoor ATtacks for Concept Bottleneck Models [8.236058439213473]
  Concept Bottleneck Models (CBMs) have emerged as a key approach to improve interpretability by leveraging high-level semantic information.<n>CBMs are susceptible to security threats, particularly backdoor attacks, which can covertly manipulate model behaviors.<n>We introduce CAT (Concept-level Backdoor ATtacks), a methodology that leverages the conceptual representations within CBMs to embed triggers during training.<n>An enhanced attack pattern, CAT+, incorporates a correlation function to systematically select the most effective and stealthy concept triggers.
  arXiv  Detail & Related papers  (2024-10-07T08:14:17Z)
• Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
  We propose a novel doubly-robust instance reweighted adversarial framework. Our importance weights are obtained by optimizing the KL-divergence regularized loss function. Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
  arXiv  Detail & Related papers  (2023-08-01T06:16:18Z)
• A general framework for defining and optimizing robustness [74.67016173858497]
  We propose a rigorous and flexible framework for defining different types of robustness properties for classifiers. Our concept is based on postulates that robustness of a classifier should be considered as a property that is independent of accuracy. We develop a very general robustness framework that is applicable to any type of classification model.
  arXiv  Detail & Related papers  (2020-06-19T13:24:20Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.