DXAI: Explaining Classification by Image Decomposition

• URL: http://arxiv.org/abs/2401.00320v2
• Date: Fri, 29 Mar 2024 12:04:52 GMT
• Title: DXAI: Explaining Classification by Image Decomposition
• Authors: Elnatan Kadar, Guy Gilboa,
• Abstract summary: We propose a new way to visualize neural network classification through a decomposition-based explainable AI (DXAI) Instead of providing an explanation heatmap, our method yields a decomposition of the image into class-agnostic and class-distinct parts.
• Score: 4.013156524547072
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a new way to explain and to visualize neural network classification through a decomposition-based explainable AI (DXAI). Instead of providing an explanation heatmap, our method yields a decomposition of the image into class-agnostic and class-distinct parts, with respect to the data and chosen classifier. Following a fundamental signal processing paradigm of analysis and synthesis, the original image is the sum of the decomposed parts. We thus obtain a radically different way of explaining classification. The class-agnostic part ideally is composed of all image features which do not posses class information, where the class-distinct part is its complementary. This new visualization can be more helpful and informative in certain scenarios, especially when the attributes are dense, global and additive in nature, for instance, when colors or textures are essential for class distinction. Code is available at https://github.com/dxai2024/dxai.

Related papers

• Accurate Explanation Model for Image Classifiers using Class Association Embedding [5.378105759529487]
  We propose a generative explanation model that combines the advantages of global and local knowledge. Class association embedding (CAE) encodes each sample into a pair of separated class-associated and individual codes. Building-block coherency feature extraction algorithm is proposed that efficiently separates class-associated features from individual ones.
  arXiv  Detail & Related papers  (2024-06-12T07:41:00Z)
• Text Descriptions are Compressive and Invariant Representations for Visual Learning [63.3464863723631]
  We show that an alternative approach, in line with humans' understanding of multiple visual features per class, can provide compelling performance in the robust few-shot learning setting. In particular, we introduce a novel method, textit SLR-AVD (Sparse Logistic Regression using Augmented Visual Descriptors). This method first automatically generates multiple visual descriptions of each class via a large language model (LLM), then uses a VLM to translate these descriptions to a set of visual feature embeddings of each image, and finally uses sparse logistic regression to select a relevant subset of these features to classify
  arXiv  Detail & Related papers  (2023-07-10T03:06:45Z)
• Why do CNNs excel at feature extraction? A mathematical explanation [53.807657273043446]
  We introduce a novel model for image classification, based on feature extraction, that can be used to generate images resembling real-world datasets. In our proof, we construct piecewise linear functions that detect the presence of features, and show that they can be realized by a convolutional network.
  arXiv  Detail & Related papers  (2023-07-03T10:41:34Z)
• Additive Class Distinction Maps using Branched-GANs [7.176107039687231]
  We present a new model, training procedure and architecture to create precise maps of distinction between two classes of images. Our proposed architecture is based on image decomposition, where the output is the sum of multiple generative networks (branched-GANs) This approach allows clear, precise and interpretable visualization of the unique characteristics of each class.
  arXiv  Detail & Related papers  (2023-05-04T15:09:54Z)
• Extracting Semantic Knowledge from GANs with Unsupervised Learning [65.32631025780631]
  Generative Adversarial Networks (GANs) encode semantics in feature maps in a linearly separable form. We propose a novel clustering algorithm, named KLiSH, which leverages the linear separability to cluster GAN's features. KLiSH succeeds in extracting fine-grained semantics of GANs trained on datasets of various objects.
  arXiv  Detail & Related papers  (2022-11-30T03:18:16Z)
• Visual Recognition with Deep Nearest Centroids [57.35144702563746]
  We devise deep nearest centroids (DNC), a conceptually elegant yet surprisingly effective network for large-scale visual recognition. Compared with parametric counterparts, DNC performs better on image classification (CIFAR-10, ImageNet) and greatly boots pixel recognition (ADE20K, Cityscapes)
  arXiv  Detail & Related papers  (2022-09-15T15:47:31Z)
• Deep ensembles in bioimage segmentation [74.01883650587321]
  In this work, we propose an ensemble of convolutional neural networks (CNNs) In ensemble methods, many different models are trained and then used for classification, the ensemble aggregates the outputs of the single classifiers. The proposed ensemble is implemented by combining different backbone networks using the DeepLabV3+ and HarDNet environment.
  arXiv  Detail & Related papers  (2021-12-24T05:54:21Z)
• Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation [84.1985497426083]
  convolutional neural networks are ill-equipped for incremental learning. New classes are available but the initial training data is not retained. We try to address this issue by "inverting" the trained segmentation network to synthesize input images starting from random noise.
  arXiv  Detail & Related papers  (2021-04-02T03:47:16Z)
• Synthesize-It-Classifier: Learning a Generative Classifier through RecurrentSelf-analysis [9.029985847202667]
  We show the generative capability of an image classifier network by synthesizing high-resolution, photo-realistic, and diverse images at scale. The overall methodology, called Synthesize-It-Classifier (STIC), does not require an explicit generator network to estimate the density of the data distribution. We demonstrate an Attentive-STIC network that shows an iterative drawing of synthesized images on the ImageNet dataset.
  arXiv  Detail & Related papers  (2021-03-26T02:00:29Z)
• Understanding invariance via feedforward inversion of discriminatively trained classifiers [30.23199531528357]
  Past research has discovered that some extraneous visual detail remains in the output logits. We develop a feedforward inversion model that produces remarkably high fidelity reconstructions. Our approach is based on BigGAN, with conditioning on logits instead of one-hot class labels.
  arXiv  Detail & Related papers  (2021-03-15T17:56:06Z)
• A causal view of compositional zero-shot recognition [42.63916938252048]
  People easily recognize new visual categories that are new combinations of known components. This compositional generalization capacity is critical for learning in real-world domains like vision and language. Here we describe an approach for compositional generalization that builds on causal ideas.
  arXiv  Detail & Related papers  (2020-06-25T17:51:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.