Human-Aided Saliency Maps Improve Generalization of Deep Learning

• URL: http://arxiv.org/abs/2105.03492v1
• Date: Fri, 7 May 2021 20:24:40 GMT
• Title: Human-Aided Saliency Maps Improve Generalization of Deep Learning
• Abstract summary: We explore encoding human judgement about salient regions of images into training data. We compare the accuracy and generalization of a state-of-the-art deep learning algorithm for a difficult problem in biometric presentation attack detection. The latter approach results in models that achieve higher accuracy and better generalization, decreasing the error of the LivDet-Iris 2020 winner from 29.78% to 16.37%.
• Score: 5.723697351415206
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning has driven remarkable accuracy increases in many computer vision problems. One ongoing challenge is how to achieve the greatest accuracy in cases where training data is limited. A second ongoing challenge is that trained models are sometimes fragile in the sense that the accuracy achieved does not generalize well, even to new data that is subjectively similar to the training set. We address these challenges in a novel way, with the first-ever (to our knowledge) exploration of encoding human judgement about salient regions of images into the training data. We compare the accuracy and generalization of a state-of-the-art deep learning algorithm for a difficult problem in biometric presentation attack detection when trained on (a) original images with typical data augmentations, and (b) the same original images transformed to encode human judgement about salient image regions. The latter approach results in models that achieve higher accuracy and better generalization, decreasing the error of the LivDet-Iris 2020 winner from 29.78% to 16.37%, and achieving impressive generalization in a leave-one-attack-type-out evaluation scenario. This work opens a new area of study for how to embed human intelligence into training strategies for deep learning to achieve high accuracy and generalization in cases of limited training data.

Related papers

• Camera trap classification with deep learning under ground truth uncertainty [2.560431498361128]
  Training labels are commonly derived from volunteer citizen science projects.<n>We investigated the effects of training under higher ground truth uncertainty.<n>Our findings have implications for applying deep learning across ecological image types with multiple labels.
  arXiv  Detail & Related papers  (2026-08-31T13:45:23Z)
• Generative AI Literacy Training Improves Intelligence Analysts' Discrimination of Real and AI-Generated Images [16.768244153862213]
  We investigate the efficacy of a brief training intervention for intelligence analysts employed by the U.S. government in 2024.<n>We find training increased overall accuracy by 9 percentage points (95% CI: [2.7, 15.4]) from a baseline of 72%.<n>The improvement is driven by a 14.2 percentage point increase in accuracy for real images.
  arXiv  Detail & Related papers  (2026-06-26T18:05:08Z)
• Combating Pattern and Content Bias: Adversarial Feature Learning for Generalized AI-Generated Image Detection [44.91910921880288]
  We propose a Multi-dimensional Adversarial Feature Learning framework to create high-quality fake images.<n>By suppressing generation-pattern and content biases, MAFL guides the model to focus on the generative features shared across different generative models.<n>Our method outperforms existing state-of-the-art approaches by 10.89% in accuracy and 8.57% in Average Precision.
  arXiv  Detail & Related papers  (2026-04-14T06:41:39Z)
• Mimicking Human Visual Development for Learning Robust Image Representations [4.210503867117539]
  We propose a progressive blurring curriculum to improve the generalization and robustness of convolutional neural networks.<n>Our approach encourages the network to prioritize global structures over high-frequency artifacts, improving robustness against distribution shifts and noisy inputs.<n>Our experiments demonstrate that the proposed curriculum reduces mean corruption error (mCE) by up to 8.30% on CIFAR-10-C and 4.43% on ImageNet-100-C datasets.
  arXiv  Detail & Related papers  (2025-12-16T12:41:04Z)
• Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarios [54.07895223545793]
  This paper introduces the Real-World Robustness dataset (RRDataset) for comprehensive evaluation of detection models across three dimensions.<n>RRDataset includes high-quality images from seven major scenarios.<n>We benchmarked 17 detectors and 10 vision-language models (VLMs) on RRDataset and conducted a large-scale human study.
  arXiv  Detail & Related papers  (2025-09-11T06:15:52Z)
• Knowledge-Guided Prompt Learning for Deepfake Facial Image Detection [54.26588902144298]
  We propose a knowledge-guided prompt learning method for deepfake facial image detection. Specifically, we retrieve forgery-related prompts from large language models as expert knowledge to guide the optimization of learnable prompts. Our proposed approach notably outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-01-01T02:18:18Z)
• What Do Learning Dynamics Reveal About Generalization in LLM Reasoning? [83.83230167222852]
  We find that a model's generalization behavior can be effectively characterized by a training metric we call pre-memorization train accuracy. By connecting a model's learning behavior to its generalization, pre-memorization train accuracy can guide targeted improvements to training strategies.
  arXiv  Detail & Related papers  (2024-11-12T09:52:40Z)
• Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness [22.856411783450284]
  We show that Dynamic Sparse Training can consistently outperform Dense Training in terms of robustness accuracy. Our findings reveal a new yet-unknown benefit of Dynamic Sparse Training and open new possibilities.
  arXiv  Detail & Related papers  (2024-10-03T22:24:54Z)
• MENTOR: Human Perception-Guided Pretraining for Increased Generalization [5.596752018167751]
  We introduce MENTOR (huMan pErceptioN-guided preTraining fOr increased geneRalization) We train an autoencoder to learn human saliency maps given an input image, without class labels. We remove the decoder part, add a classification layer on top of the encoder, and fine-tune this new model conventionally.
  arXiv  Detail & Related papers  (2023-10-30T13:50:44Z)
• Rapid Training Data Creation by Synthesizing Medical Images for Classification and Localization [10.506584969668792]
  We present a method for the transformation of real data to train any Deep Neural Network to solve the above problems. For the weakly supervised model, we show that the localization accuracy increases significantly using the generated data. In the latter model, we show that the accuracy, when trained with generated images, closely parallels the accuracy when trained with exhaustively annotated real images.
  arXiv  Detail & Related papers  (2023-08-09T03:49:12Z)
• Pruning Distorted Images in MNIST Handwritten Digits [0.0]
  We propose a two-stage deep learning approach to recognize handwritten digits. In the first stage, we create a simple neural network to identify distorted digits within the training set. In the second stage, we exclude these identified images from the training dataset and proceed to retrain the model using the filtered dataset. Our experimental results demonstrate the effectiveness of the proposed approach, achieving an accuracy rate of over 99.5% on the testing dataset.
  arXiv  Detail & Related papers  (2023-05-26T11:44:35Z)
• CutPaste: Self-Supervised Learning for Anomaly Detection and Localization [59.719925639875036]
  We propose a framework for building anomaly detectors using normal training data only. We first learn self-supervised deep representations and then build a generative one-class classifier on learned representations. Our empirical study on MVTec anomaly detection dataset demonstrates the proposed algorithm is general to be able to detect various types of real-world defects.
  arXiv  Detail & Related papers  (2021-04-08T19:04:55Z)
• Pre-training strategies and datasets for facial representation learning [58.8289362536262]
  We show how to find a universal face representation that can be adapted to several facial analysis tasks and datasets. We systematically investigate two ways of large-scale representation learning applied to faces: supervised and unsupervised pre-training. Our main two findings are: Unsupervised pre-training on completely in-the-wild, uncurated data provides consistent and, in some cases, significant accuracy improvements.
  arXiv  Detail & Related papers  (2021-03-30T17:57:25Z)
• Deepfake Forensics via An Adversarial Game [99.84099103679816]
  We advocate adversarial training for improving the generalization ability to both unseen facial forgeries and unseen image/video qualities. Considering that AI-based face manipulation often leads to high-frequency artifacts that can be easily spotted by models yet difficult to generalize, we propose a new adversarial training method that attempts to blur out these specific artifacts.
  arXiv  Detail & Related papers  (2021-03-25T02:20:08Z)
• Data optimization for large batch distributed training of deep neural networks [0.19336815376402716]
  Current practice for distributed training of deep neural networks faces the challenges of communication bottlenecks when operating at scale. We propose a data optimization approach that utilize machine learning to implicitly smooth out the loss landscape resulting in fewer local minima. Our approach filters out data points which are less important to feature learning, enabling us to speed up the training of models on larger batch sizes to improved accuracy.
  arXiv  Detail & Related papers  (2020-12-16T21:22:02Z)
• Deep Traffic Sign Detection and Recognition Without Target Domain Real Images [52.079665469286496]
  We propose a novel database generation method that requires no real image from the target-domain, and (ii) templates of the traffic signs. The method does not aim at overcoming the training with real data, but to be a compatible alternative when the real data is not available. On large data sets, training with a fully synthetic data set almost matches the performance of training with a real one.
  arXiv  Detail & Related papers  (2020-07-30T21:06:47Z)
• Semi-Siamese Training for Shallow Face Learning [78.7386209619276]
  We introduce a novel training method named Semi-Siamese Training (SST) A pair of Semi-Siamese networks constitute the forward propagation structure, and the training loss is computed with an updating gallery queue. Our method is developed without extra-dependency, thus can be flexibly integrated with the existing loss functions and network architectures.
  arXiv  Detail & Related papers  (2020-07-16T15:20:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.