Deep Internal Learning: Deep Learning from a Single Input

• URL: http://arxiv.org/abs/2312.07425v2
• Date: Mon, 8 Apr 2024 16:56:17 GMT
• Title: Deep Internal Learning: Deep Learning from a Single Input
• Authors: Tom Tirer, Raja Giryes, Se Young Chun, Yonina C. Eldar,
• Abstract summary: In many cases there is value in training a network just from the input at hand. This is particularly relevant in many signal and image processing problems where training data is scarce and diversity is large. This survey paper aims at covering deep internal-learning techniques that have been proposed in the past few years for these two important directions.
• Score: 88.59966585422914
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning, in general, focuses on training a neural network from large labeled datasets. Yet, in many cases there is value in training a network just from the input at hand. This is particularly relevant in many signal and image processing problems where training data is scarce and diversity is large on the one hand, and on the other, there is a lot of structure in the data that can be exploited. Using this information is the key to deep internal-learning strategies, which may involve training a network from scratch using a single input or adapting an already trained network to a provided input example at inference time. This survey paper aims at covering deep internal-learning techniques that have been proposed in the past few years for these two important directions. While our main focus will be on image processing problems, most of the approaches that we survey are derived for general signals (vectors with recurring patterns that can be distinguished from noise) and are therefore applicable to other modalities.

Related papers

• Opening the Black Box: predicting the trainability of deep neural networks with reconstruction entropy [0.0]
  We present a method for predicting the trainable regime in parameter space for deep feedforward neural networks. For both the MNIST and CIFAR10 datasets, we show that a single epoch of training is sufficient to predict the trainability of the deep feedforward network.
  arXiv  Detail & Related papers  (2024-06-13T18:00:05Z)
• Uncertainty Quantification and Resource-Demanding Computer Vision Applications of Deep Learning [5.130440339897478]
  Bringing deep neural networks (DNNs) into safety critical applications requires a thorough treatment of the model's uncertainties. In this article, we survey methods that we developed to teach DNNs to be uncertain when they encounter new object classes. We also present training methods to learn from only a few labels with help of uncertainty quantification.
  arXiv  Detail & Related papers  (2022-05-30T08:31:03Z)
• Being Friends Instead of Adversaries: Deep Networks Learn from Data Simplified by Other Networks [23.886422706697882]
  A different idea has been recently proposed, named Friendly Training, which consists in altering the input data by adding an automatically estimated perturbation. We revisit and extend this idea inspired by the effectiveness of neural generators in the context of Adversarial Machine Learning. We propose an auxiliary multi-layer network that is responsible of altering the input data to make them easier to be handled by the classifier.
  arXiv  Detail & Related papers  (2021-12-18T16:59:35Z)
• Reasoning-Modulated Representations [85.08205744191078]
  We study a common setting where our task is not purely opaque. Our approach paves the way for a new class of data-efficient representation learning.
  arXiv  Detail & Related papers  (2021-07-19T13:57:13Z)
• Multi-Agent Semi-Siamese Training for Long-tail and Shallow Face Learning [54.13876727413492]
  In many real-world scenarios of face recognition, the depth of training dataset is shallow, which means only two face images are available for each ID. With the non-uniform increase of samples, such issue is converted to a more general case, a.k.a a long-tail face learning. Based on the Semi-Siamese Training (SST), we introduce an advanced solution, named Multi-Agent Semi-Siamese Training (MASST) MASST includes a probe network and multiple gallery agents, the former aims to encode the probe features, and the latter constitutes a stack of
  arXiv  Detail & Related papers  (2021-05-10T04:57:32Z)
• A neural anisotropic view of underspecification in deep learning [60.119023683371736]
  We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
  arXiv  Detail & Related papers  (2021-04-29T14:31:09Z)
• Federated Learning: A Signal Processing Perspective [144.63726413692876]
  Federated learning is an emerging machine learning paradigm for training models across multiple edge devices holding local datasets, without explicitly exchanging the data. This article provides a unified systematic framework for federated learning in a manner that encapsulates and highlights the main challenges that are natural to treat using signal processing tools.
  arXiv  Detail & Related papers  (2021-03-31T15:14:39Z)
• Laplacian Denoising Autoencoder [114.21219514831343]
  We propose to learn data representations with a novel type of denoising autoencoder. The noisy input data is generated by corrupting latent clean data in the gradient domain. Experiments on several visual benchmarks demonstrate that better representations can be learned with the proposed approach.
  arXiv  Detail & Related papers  (2020-03-30T16:52:39Z)
• When Deep Learning Meets Data Alignment: A Review on Deep Registration Networks (DRNs) [4.616914111718527]
  Recent advancements in machine learning could be a turning point in the field of computer vision. Recent advancements in machine learning could be a turning point in the field of computer vision.
  arXiv  Detail & Related papers  (2020-03-06T12:56:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.