Validation and generalization of pixel-wise relevance in convolutional neural networks trained for face classification

• URL: http://arxiv.org/abs/2006.16795v1
• Date: Tue, 16 Jun 2020 23:20:40 GMT
• Title: Validation and generalization of pixel-wise relevance in convolutional neural networks trained for face classification
• Authors: J\~nani Crawford, Eshed Margalit, Kalanit Grill-Spector, and Sonia Poltoratski
• Abstract summary: We show how relevance measures vary with and generalize across key model parameters. Using relevance-based image masking, we find that relevance maps for face classification prove generally stable. Fine-grained analyses of relevance maps across models revealed asymmetries in generalization that point to specific benefits of choice parameters.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The increased use of convolutional neural networks for face recognition in science, governance, and broader society has created an acute need for methods that can show how these 'black box' decisions are made. To be interpretable and useful to humans, such a method should convey a model's learned classification strategy in a way that is robust to random initializations or spurious correlations in input data. To this end, we applied the decompositional pixel-wise attribution method of layer-wise relevance propagation (LRP) to resolve the decisions of several classes of VGG-16 models trained for face recognition. We then quantified how these relevance measures vary with and generalize across key model parameters, such as the pretraining dataset (ImageNet or VGGFace), the finetuning task (gender or identity classification), and random initializations of model weights. Using relevance-based image masking, we find that relevance maps for face classification prove generally stable across random initializations, and can generalize across finetuning tasks. However, there is markedly less generalization across pretraining datasets, indicating that ImageNet- and VGGFace-trained models sample face information differently even as they achieve comparably high classification performance. Fine-grained analyses of relevance maps across models revealed asymmetries in generalization that point to specific benefits of choice parameters, and suggest that it may be possible to find an underlying set of important face image pixels that drive decisions across convolutional neural networks and tasks. Finally, we evaluated model decision weighting against human measures of similarity, providing a novel framework for interpreting face recognition decisions across human and machine.

Related papers

• COMIX: Compositional Explanations using Prototypes [46.15031477955461]
  We propose a method to align machine representations with human understanding. The proposed method, named COMIX, classifies an image by decomposing it into regions based on learned concepts. We show that our method provides fidelity of explanations and shows that the efficiency is competitive with other inherently interpretable architectures.
  arXiv  Detail & Related papers  (2025-01-10T15:40:31Z)
• An Efficient Framework for Enhancing Discriminative Models via Diffusion Techniques [12.470257882838126]
  We propose the Diffusion-Based Discriminative Model Enhancement Framework (DBMEF) This framework seamlessly integrates discriminative and generative models in a training-free manner. DBMEF can effectively enhance the classification accuracy and capability of discriminative models in a plug-and-play manner.
  arXiv  Detail & Related papers  (2024-12-12T08:46:22Z)
• InfoDisent: Explainability of Image Classification Models by Information Disentanglement [9.380255522558294]
  We introduce InfoDisent, a hybrid model that combines the advantages of both approaches. By utilizing an information bottleneck, InfoDisent disentangles the information in the final layer of a pre-trained deep network. We validate the effectiveness of InfoDisent on benchmark datasets such as ImageNet, CUB-200-2011, Stanford Cars, and Stanford Dogs.
  arXiv  Detail & Related papers  (2024-09-16T14:39:15Z)
• Reinforcing Pre-trained Models Using Counterfactual Images [54.26310919385808]
  This paper proposes a novel framework to reinforce classification models using language-guided generated counterfactual images. We identify model weaknesses by testing the model using the counterfactual image dataset. We employ the counterfactual images as an augmented dataset to fine-tune and reinforce the classification model.
  arXiv  Detail & Related papers  (2024-06-19T08:07:14Z)
• Neural Clustering based Visual Representation Learning [61.72646814537163]
  Clustering is one of the most classic approaches in machine learning and data analysis. We propose feature extraction with clustering (FEC), which views feature extraction as a process of selecting representatives from data. FEC alternates between grouping pixels into individual clusters to abstract representatives and updating the deep features of pixels with current representatives.
  arXiv  Detail & Related papers  (2024-03-26T06:04:50Z)
• Appearance Debiased Gaze Estimation via Stochastic Subject-Wise Adversarial Learning [33.55397868171977]
  Appearance-based gaze estimation has been attracting attention in computer vision, and remarkable improvements have been achieved using various deep learning techniques. We propose a novel framework: subject-wise gaZE learning (SAZE), which trains a network to generalize the appearance of subjects. Our experimental results verify the robustness of the method in that it yields state-of-the-art performance, achieving 3.89 and 4.42 on the MPIIGaze and EyeDiap datasets, respectively.
  arXiv  Detail & Related papers  (2024-01-25T00:23:21Z)
• Fine-grained Recognition with Learnable Semantic Data Augmentation [68.48892326854494]
  Fine-grained image recognition is a longstanding computer vision challenge. We propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Our method significantly improves the generalization performance on several popular classification networks.
  arXiv  Detail & Related papers  (2023-09-01T11:15:50Z)
• Learning Contrastive Representation for Semantic Correspondence [150.29135856909477]
  We propose a multi-level contrastive learning approach for semantic matching. We show that image-level contrastive learning is a key component to encourage the convolutional features to find correspondence between similar objects.
  arXiv  Detail & Related papers  (2021-09-22T18:34:14Z)
• Visual Recognition with Deep Learning from Biased Image Datasets [6.10183951877597]
  We show how biasing models can be applied to remedy problems in the context of visual recognition. Based on the (approximate) knowledge of the biasing mechanisms at work, our approach consists in reweighting the observations. We propose to use a low dimensional image representation, shared across the image databases.
  arXiv  Detail & Related papers  (2021-09-06T10:56:58Z)
• Learning to Learn Parameterized Classification Networks for Scalable Input Images [76.44375136492827]
  Convolutional Neural Networks (CNNs) do not have a predictable recognition behavior with respect to the input resolution change. We employ meta learners to generate convolutional weights of main networks for various input scales. We further utilize knowledge distillation on the fly over model predictions based on different input resolutions.
  arXiv  Detail & Related papers  (2020-07-13T04:27:25Z)
• Pre-Trained Models for Heterogeneous Information Networks [57.78194356302626]
  We propose a self-supervised pre-training and fine-tuning framework, PF-HIN, to capture the features of a heterogeneous information network. PF-HIN consistently and significantly outperforms state-of-the-art alternatives on each of these tasks, on four datasets.
  arXiv  Detail & Related papers  (2020-07-07T03:36:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.