Achieving Data Efficient Neural Networks with Hybrid Concept-based Models

• URL: http://arxiv.org/abs/2408.07438v1
• Date: Wed, 14 Aug 2024 10:15:34 GMT
• Title: Achieving Data Efficient Neural Networks with Hybrid Concept-based Models
• Authors: Tobias A. Opsahl, Vegard Antun,
• Abstract summary: We introduce two novel model architectures that train using both class labels and additional information in the dataset referred to as concepts. We show that the hybrid concept-based models outperform standard computer vision models with respect to accuracy, especially in sparse data settings. We also introduce an algorithm for performing adversarial concept attacks, where an image is perturbed in a way that does not change a concept-based model's concept predictions, but changes the class prediction.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most datasets used for supervised machine learning consist of a single label per data point. However, in cases where more information than just the class label is available, would it be possible to train models more efficiently? We introduce two novel model architectures, which we call hybrid concept-based models, that train using both class labels and additional information in the dataset referred to as concepts. In order to thoroughly assess their performance, we introduce ConceptShapes, an open and flexible class of datasets with concept labels. We show that the hybrid concept-based models outperform standard computer vision models and previously proposed concept-based models with respect to accuracy, especially in sparse data settings. We also introduce an algorithm for performing adversarial concept attacks, where an image is perturbed in a way that does not change a concept-based model's concept predictions, but changes the class prediction. The existence of such adversarial examples raises questions about the interpretable qualities promised by concept-based models.

Related papers

• Visual Data Diagnosis and Debiasing with Concept Graphs [50.84781894621378]
  Deep learning models often pick up inherent biases in the data during the training process, leading to unreliable predictions. We present CONBIAS, a novel framework for diagnosing and mitigating Concept co-occurrence Biases in visual datasets. We show that by employing a novel clique-based concept balancing strategy, we can mitigate these imbalances, leading to enhanced performance on downstream tasks.
  arXiv  Detail & Related papers  (2024-09-26T16:59:01Z)
• Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery [52.498055901649025]
  Concept Bottleneck Models (CBMs) have been proposed to address the 'black-box' problem of deep neural networks. We propose a novel CBM approach -- called Discover-then-Name-CBM (DN-CBM) -- that inverts the typical paradigm. Our concept extraction strategy is efficient, since it is agnostic to the downstream task, and uses concepts already known to the model.
  arXiv  Detail & Related papers  (2024-07-19T17:50:11Z)
• Concept Bottleneck Models Without Predefined Concepts [26.156636891713745]
  We introduce an input-dependent concept selection mechanism that ensures only a small subset of concepts is used across all classes. We show that our approach improves downstream performance and narrows the performance gap to black-box models.
  arXiv  Detail & Related papers  (2024-07-04T13:34:50Z)
• Restyling Unsupervised Concept Based Interpretable Networks with Generative Models [14.604305230535026]
  We propose a novel method that relies on mapping the concept features to the latent space of a pretrained generative model. We quantitatively ascertain the efficacy of our method in terms of accuracy of the interpretable prediction network, fidelity of reconstruction, as well as faithfulness and consistency of learnt concepts.
  arXiv  Detail & Related papers  (2024-07-01T14:39:41Z)
• Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learning [86.15009879251386]
  We propose a novel architecture and method of explainable classification with Concept Bottleneck Models (CBM) CBMs require an additional set of concepts to leverage. We show a significant increase in accuracy using sparse hidden layers in CLIP-based bottleneck models.
  arXiv  Detail & Related papers  (2024-04-04T09:43:43Z)
• Attributing Learned Concepts in Neural Networks to Training Data [5.930268338525991]
  We find evidence for convergence, where removing the 10,000 top attributing images for a concept and retraining the model does not change the location of the concept in the network. This suggests that the features that inform the development of a concept are spread in a more diffuse manner across its exemplars, implying robustness in concept formation.
  arXiv  Detail & Related papers  (2023-10-04T20:26:59Z)
• Synthetic Model Combination: An Instance-wise Approach to Unsupervised Ensemble Learning [92.89846887298852]
  Consider making a prediction over new test data without any opportunity to learn from a training set of labelled data. Give access to a set of expert models and their predictions alongside some limited information about the dataset used to train them.
  arXiv  Detail & Related papers  (2022-10-11T10:20:31Z)
• ConceptDistil: Model-Agnostic Distillation of Concept Explanations [4.462334751640166]
  Concept-based explanations aims to fill the model interpretability gap for non-technical humans-in-the-loop. We propose ConceptDistil, a method to bring concept explanations to any black-box classifier using knowledge distillation. We validate ConceptDistil in a real world use-case, showing that it is able to optimize both tasks.
  arXiv  Detail & Related papers  (2022-05-07T08:58:54Z)
• Concept Bottleneck Model with Additional Unsupervised Concepts [0.5939410304994348]
  We propose a novel interpretable model based on the concept bottleneck model (CBM) CBM uses concept labels to train an intermediate layer as the additional visible layer. By seamlessly training these two types of concepts while reducing the amount of computation, we can obtain both supervised and unsupervised concepts simultaneously.
  arXiv  Detail & Related papers  (2022-02-03T08:30:51Z)
• What do we expect from Multiple-choice QA Systems? [70.86513724662302]
  We consider a top performing model on several Multiple Choice Question Answering (MCQA) datasets. We evaluate it against a set of expectations one might have from such a model, using a series of zero-information perturbations of the model's inputs.
  arXiv  Detail & Related papers  (2020-11-20T21:27:10Z)
• Concept Bottleneck Models [79.91795150047804]
  State-of-the-art models today do not typically support the manipulation of concepts like "the existence of bone spurs" We revisit the classic idea of first predicting concepts that are provided at training time, and then using these concepts to predict the label. On x-ray grading and bird identification, concept bottleneck models achieve competitive accuracy with standard end-to-end models.
  arXiv  Detail & Related papers  (2020-07-09T07:47:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.