DISentangled Counterfactual Visual interpretER (DISCOVER) generalizes to natural images
- URL: http://arxiv.org/abs/2406.15918v1
- Date: Sat, 22 Jun 2024 19:05:50 GMT
- Title: DISentangled Counterfactual Visual interpretER (DISCOVER) generalizes to natural images
- Authors: Oded Rotem, Assaf Zaritsky,
- Abstract summary: We show that DISentangled COunterfactual Visual interpretER (DISCOVER) can be applied to the domain of natural images.
First, DISCOVER visually interpreted the nose size, the muzzle area, and the face size as semantic discriminative visual traits discriminating between facial images of dogs versus cats.
Second, DISCOVER visually interpreted the cheeks and jawline, eyebrows and hair, and the eyes, as discriminative facial characteristics.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: We recently presented DISentangled COunterfactual Visual interpretER (DISCOVER), a method toward systematic visual interpretability of image-based classification models and demonstrated its applicability to two biomedical domains. Here we demonstrate that DISCOVER can be applied to the domain of natural images. First, DISCOVER visually interpreted the nose size, the muzzle area, and the face size as semantic discriminative visual traits discriminating between facial images of dogs versus cats. Second, DISCOVER visually interpreted the cheeks and jawline, eyebrows and hair, and the eyes, as discriminative facial characteristics. These successful visual interpretations across two natural images domains indicate that DISCOVER is a generalized interpretability method.
Related papers
- For a semiotic AI: Bridging computer vision and visual semiotics for computational observation of large scale facial image archives [3.418398936676879]
This work presents FRESCO, a framework designed to explore the socio-cultural implications of images on social media platforms at scale.
FRESCO deconstructs images into numerical and categorical variables using state-of-the-art computer vision techniques.
The framework analyzes images across three levels: the plastic level, encompassing fundamental visual features like lines and colors; the figurative level, representing specific entities or concepts; and the enunciation level, which focuses particularly on constructing the point of view of the spectator and observer.
arXiv Detail & Related papers (2024-07-03T16:57:38Z) - Explaining Representation Learning with Perceptual Components [14.10876324116018]
Self-supervised models create representation spaces that lack clear semantic meaning.
We introduce a novel method to analyze representation spaces using three key perceptual components: color, shape, and texture.
Our approach enhances the interpretability of the representation space, offering explanations that resonate with human visual perception.
arXiv Detail & Related papers (2024-06-11T04:08:37Z) - Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning [64.1316997189396]
We present a novel language-tied self-supervised learning framework, Hierarchical Language-tied Self-Supervision (HLSS) for histopathology images.
Our resulting model achieves state-of-the-art performance on two medical imaging benchmarks, OpenSRH and TCGA datasets.
arXiv Detail & Related papers (2024-03-21T17:58:56Z) - Impressions: Understanding Visual Semiotics and Aesthetic Impact [66.40617566253404]
We present Impressions, a novel dataset through which to investigate the semiotics of images.
We show that existing multimodal image captioning and conditional generation models struggle to simulate plausible human responses to images.
This dataset significantly improves their ability to model impressions and aesthetic evaluations of images through fine-tuning and few-shot adaptation.
arXiv Detail & Related papers (2023-10-27T04:30:18Z) - Multi-Domain Norm-referenced Encoding Enables Data Efficient Transfer
Learning of Facial Expression Recognition [62.997667081978825]
We propose a biologically-inspired mechanism for transfer learning in facial expression recognition.
Our proposed architecture provides an explanation for how the human brain might innately recognize facial expressions on varying head shapes.
Our model achieves a classification accuracy of 92.15% on the FERG dataset with extreme data efficiency.
arXiv Detail & Related papers (2023-04-05T09:06:30Z) - PERI: Part Aware Emotion Recognition In The Wild [4.206175795966693]
This paper focuses on emotion recognition using visual features.
We create part aware spatial (PAS) images by extracting key regions from the input image using a mask generated from both body pose and facial landmarks.
We provide our results on the publicly available in the wild EMOTIC dataset.
arXiv Detail & Related papers (2022-10-18T20:01:40Z) - A domain adaptive deep learning solution for scanpath prediction of
paintings [66.46953851227454]
This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings.
We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans.
The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
arXiv Detail & Related papers (2022-09-22T22:27:08Z) - Prune and distill: similar reformatting of image information along rat
visual cortex and deep neural networks [61.60177890353585]
Deep convolutional neural networks (CNNs) have been shown to provide excellent models for its functional analogue in the brain, the ventral stream in visual cortex.
Here we consider some prominent statistical patterns that are known to exist in the internal representations of either CNNs or the visual cortex.
We show that CNNs and visual cortex share a similarly tight relationship between dimensionality expansion/reduction of object representations and reformatting of image information.
arXiv Detail & Related papers (2022-05-27T08:06:40Z) - Distinguishing Natural and Computer-Generated Images using
Multi-Colorspace fused EfficientNet [0.0]
In a real-world image forensic scenario, it is highly essential to consider all categories of image generation.
We propose a Multi-Colorspace fused EfficientNet model by parallelly fusing three EfficientNet networks.
Our model outperforms the baselines in terms of accuracy, robustness towards post-processing, and generalizability towards other datasets.
arXiv Detail & Related papers (2021-10-18T15:55:45Z) - InterFaceGAN: Interpreting the Disentangled Face Representation Learned
by GANs [73.27299786083424]
We propose a framework called InterFaceGAN to interpret the disentangled face representation learned by state-of-the-art GAN models.
We first find that GANs learn various semantics in some linear subspaces of the latent space.
We then conduct a detailed study on the correlation between different semantics and manage to better disentangle them via subspace projection.
arXiv Detail & Related papers (2020-05-18T18:01:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.