Self-Denoising Neural Networks for Few Shot Learning

• URL: http://arxiv.org/abs/2110.13386v1
• Date: Tue, 26 Oct 2021 03:28:36 GMT
• Title: Self-Denoising Neural Networks for Few Shot Learning
• Authors: Steven Schwarcz, Sai Saketh Rambhatla, Rama Chellappa
• Abstract summary: We present a new training scheme that adds noise at multiple stages of an existing neural architecture while simultaneously learning to be robust to this added noise. This architecture, which we call a Self-Denoising Neural Network (SDNN), can be applied easily to most modern convolutional neural architectures.
• Score: 66.38505903102373
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we introduce a new architecture for few shot learning, the task of teaching a neural network from as few as one or five labeled examples. Inspired by the theoretical results of Alaine et al that Denoising Autoencoders refine features to lie closer to the true data manifold, we present a new training scheme that adds noise at multiple stages of an existing neural architecture while simultaneously learning to be robust to this added noise. This architecture, which we call a Self-Denoising Neural Network (SDNN), can be applied easily to most modern convolutional neural architectures, and can be used as a supplement to many existing few-shot learning techniques. We empirically show that SDNNs out-perform previous state-of-the-art methods for few shot image recognition using the Wide-ResNet architecture on the \textit{mini}ImageNet, tiered-ImageNet, and CIFAR-FS few shot learning datasets. We also perform a series of ablation experiments to empirically justify the construction of the SDNN architecture. Finally, we show that SDNNs even improve few shot performance on the task of human action detection in video using experiments on the ActEV SDL Surprise Activities challenge.

Related papers

• DQNAS: Neural Architecture Search using Reinforcement Learning [6.33280703577189]
  Convolutional Neural Networks have been used in a variety of image related applications. In this paper, we propose an automated Neural Architecture Search framework, guided by the principles of Reinforcement Learning.
  arXiv  Detail & Related papers  (2023-01-17T04:01:47Z)
• Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis [148.16279746287452]
  We propose a swin-conv block to incorporate the local modeling ability of residual convolutional layer and non-local modeling ability of swin transformer block. For the training data synthesis, we design a practical noise degradation model which takes into consideration different kinds of noise. Experiments on AGWN removal and real image denoising demonstrate that the new network architecture design achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-24T18:11:31Z)
• Efficient Neural Architecture Search with Performance Prediction [0.0]
  We use a neural architecture search to find the best network architecture for the task at hand. Existing NAS algorithms generally evaluate the fitness of a new architecture by fully training from scratch. An end-to-end offline performance predictor is proposed to accelerate the evaluation of sampled architectures.
  arXiv  Detail & Related papers  (2021-08-04T05:44:16Z)
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search [65.79109790446257]
  Recent work has shown that the structure of deep convolutional neural networks can be used as a structured image prior. We propose to search for neural architectures that capture stronger image priors. We search for an improved network by leveraging an existing neural architecture search algorithm.
  arXiv  Detail & Related papers  (2020-08-26T17:59:36Z)
• Attentive Graph Neural Networks for Few-Shot Learning [74.01069516079379]
  Graph Neural Networks (GNN) has demonstrated the superior performance in many challenging applications, including the few-shot learning tasks. Despite its powerful capacity to learn and generalize the model from few samples, GNN usually suffers from severe over-fitting and over-smoothing as the model becomes deep. We propose a novel Attentive GNN to tackle these challenges, by incorporating a triple-attention mechanism.
  arXiv  Detail & Related papers  (2020-07-14T07:43:09Z)
• Multi-fidelity Neural Architecture Search with Knowledge Distillation [69.09782590880367]
  We propose a bayesian multi-fidelity method for neural architecture search: MF-KD. Knowledge distillation adds to a loss function a term forcing a network to mimic some teacher network. We show that training for a few epochs with such a modified loss function leads to a better selection of neural architectures than training for a few epochs with a logistic loss.
  arXiv  Detail & Related papers  (2020-06-15T12:32:38Z)
• A Semi-Supervised Assessor of Neural Architectures [157.76189339451565]
  We employ an auto-encoder to discover meaningful representations of neural architectures. A graph convolutional neural network is introduced to predict the performance of architectures.
  arXiv  Detail & Related papers  (2020-05-14T09:02:33Z)
• Superkernel Neural Architecture Search for Image Denoising [4.633161758939184]
  We focus on exploring NAS for a dense prediction task that is image denoising. Due to a costly training procedure, most NAS solutions for image enhancement rely on reinforcement learning or evolutionary algorithm exploration. We introduce a new efficient implementation of various super Kernel techniques that enable fast single-shot training of models for dense predictions. We demonstrate the effectiveness of our method on the SIDD+ benchmark for image denoising.
  arXiv  Detail & Related papers  (2020-04-19T14:52:22Z)
• Automated Pavement Crack Segmentation Using U-Net-based Convolutional Neural Network [10.48658033897047]
  We propose a deep learning technique based on a convolutional neural network to perform segmentation tasks on pavement crack images. Our approach requires minimal feature engineering compared to other machine learning techniques. Our method achieves an F1 score of 96% on the CFD dataset and 73% on the Crack500 dataset, outperforming other algorithms tested on these datasets.
  arXiv  Detail & Related papers  (2020-01-07T07:18:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.