Image Class Translation Distance: A Novel Interpretable Feature for Image Classification

• URL: http://arxiv.org/abs/2408.08973v1
• Date: Fri, 16 Aug 2024 18:48:28 GMT
• Title: Image Class Translation Distance: A Novel Interpretable Feature for Image Classification
• Authors: Mikyla K. Bowen, Jesse W. Wilson,
• Abstract summary: We propose a novel application of image translation networks for image classification. We train a network to translate images between possible classes, and then quantify translation distance. These translation distances can then be examined for clusters and trends, and can be fed directly to a simple classifier. We demonstrate the approach on a toy 2-class scenario, apples versus oranges, and then apply it to two medical imaging tasks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We propose a novel application of image translation networks for image classification and demonstrate its potential as a more interpretable alternative to conventional black box classification networks. We train a network to translate images between possible classes, and then quantify translation distance, i.e. the degree of alteration needed to conform an image to one class or another. These translation distances can then be examined for clusters and trends, and can be fed directly to a simple classifier (e.g. a support vector machine, SVM), providing comparable accuracy compared to a conventional end-to-end convolutional neural network classifier. In addition, visual inspection of translated images can reveal class-specific characteristics and biases in the training sets, such as visual artifacts that are more frequently observed in one class or another. We demonstrate the approach on a toy 2-class scenario, apples versus oranges, and then apply it to two medical imaging tasks: detecting melanoma from photographs of pigmented lesions and classifying 6 cell types in a bone marrow biopsy smear. This novel application of image-to-image networks shows the potential of the technology to go beyond imagining different stylistic changes and to provide greater insight into image classification and medical imaging datasets.

Related papers

• PM2: A New Prompting Multi-modal Model Paradigm for Few-shot Medical Image Classification [12.628447384868503]
  We propose a new prompting multi-modal model paradigm on medical image classification based on multi-modal foundation models, called PM2. Besides image modality, PM2 introduces another supplementary text input, known as prompt, to further describe corresponding image or concept classes. Our PM2 significantly outperforms counterparts regardless of prompt schemes and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-04-13T07:27:06Z)
• An Explainable Model-Agnostic Algorithm for CNN-based Biometrics Verification [55.28171619580959]
  This paper describes an adaptation of the Local Interpretable Model-Agnostic Explanations (LIME) AI method to operate under a biometric verification setting.
  arXiv  Detail & Related papers  (2023-07-25T11:51:14Z)
• A Test Statistic Estimation-based Approach for Establishing Self-interpretable CNN-based Binary Classifiers [7.424003880270276]
  Post-hoc interpretability methods have the limitation that they can produce plausible but different interpretations. The proposed method is self-interpretable, quantitative. Unlike the traditional post-hoc interpretability methods, the proposed method is self-interpretable, quantitative.
  arXiv  Detail & Related papers  (2023-03-13T05:51:35Z)
• Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing [53.89917396428747]
  Self-supervised learning in vision-language processing exploits semantic alignment between imaging and text modalities. We explicitly account for prior images and reports when available during both training and fine-tuning. Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model.
  arXiv  Detail & Related papers  (2023-01-11T16:35:33Z)
• Traditional Classification Neural Networks are Good Generators: They are Competitive with DDPMs and GANs [104.72108627191041]
  We show that conventional neural network classifiers can generate high-quality images comparable to state-of-the-art generative models. We propose a mask-based reconstruction module to make semantic gradients-aware to synthesize plausible images. We show that our method is also applicable to text-to-image generation by regarding image-text foundation models.
  arXiv  Detail & Related papers  (2022-11-27T11:25:35Z)
• Dermoscopic Image Classification with Neural Style Transfer [5.314466196448187]
  We propose an adaptation of the Neural Style Transfer (NST) as a novel image pre-processing step for skin lesion classification problems. We represent each dermoscopic image as the style image and transfer the style of the lesion onto a homogeneous content image. This transfers the main variability of each lesion onto the same localized region, which allows us to integrate the generated images together and extract latent, low-rank style features.
  arXiv  Detail & Related papers  (2021-05-17T03:50:51Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)
• Improving Endoscopic Decision Support Systems by Translating Between Imaging Modalities [4.760079434948197]
  We investigate the applicability of image-to-image translation to endoscopic images showing different imaging modalities. In a study on computer-aided celiac disease diagnosis, we explore whether image-to-image translation is capable of effectively performing the translation between the domains.
  arXiv  Detail & Related papers  (2020-04-27T06:55:56Z)
• Melanoma Detection using Adversarial Training and Deep Transfer Learning [6.22964000148682]
  We propose a two-stage framework for automatic classification of skin lesion images. In the first stage, we leverage the inter-class variation of the data distribution for the task of conditional image synthesis. In the second stage, we train a deep convolutional neural network for skin lesion classification.
  arXiv  Detail & Related papers  (2020-04-14T22:46:20Z)
• Structural-analogy from a Single Image Pair [118.61885732829117]
  In this paper, we explore the capabilities of neural networks to understand image structure given only a single pair of images, A and B. We generate an image that keeps the appearance and style of B, but has a structural arrangement that corresponds to A. Our method can be used to generate high quality imagery in other conditional generation tasks utilizing images A and B only.
  arXiv  Detail & Related papers  (2020-04-05T14:51:10Z)
• I Am Going MAD: Maximum Discrepancy Competition for Comparing Classifiers Adaptively [135.7695909882746]
  We name the MAximum Discrepancy (MAD) competition. We adaptively sample a small test set from an arbitrarily large corpus of unlabeled images. Human labeling on the resulting model-dependent image sets reveals the relative performance of the competing classifiers.
  arXiv  Detail & Related papers  (2020-02-25T03:32:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.