Survey: Exploiting Data Redundancy for Optimization of Deep Learning
- URL: http://arxiv.org/abs/2208.13363v1
- Date: Mon, 29 Aug 2022 04:31:18 GMT
- Title: Survey: Exploiting Data Redundancy for Optimization of Deep Learning
- Authors: Jou-An Chen, Wei Niu, Bin Ren, Yanzhi Wang, Xipeng Shen
- Abstract summary: Data redundancy is ubiquitous in the inputs and intermediate results of Deep Neural Networks (DNN)
This article surveys hundreds of recent papers on the topic.
It introduces a novel taxonomy to put the various techniques into a single categorization framework.
- Score: 42.1585031880029
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Data redundancy is ubiquitous in the inputs and intermediate results of Deep
Neural Networks (DNN). It offers many significant opportunities for improving
DNN performance and efficiency and has been explored in a large body of work.
These studies have scattered in many venues across several years. The targets
they focus on range from images to videos and texts, and the techniques they
use to detect and exploit data redundancy also vary in many aspects. There is
not yet a systematic examination and summary of the many efforts, making it
difficult for researchers to get a comprehensive view of the prior work, the
state of the art, differences and shared principles, and the areas and
directions yet to explore. This article tries to fill the void. It surveys
hundreds of recent papers on the topic, introduces a novel taxonomy to put the
various techniques into a single categorization framework, offers a
comprehensive description of the main methods used for exploiting data
redundancy in improving multiple kinds of DNNs on data, and points out a set of
research opportunities for future to explore.
Related papers
- Domain Generalization through Meta-Learning: A Survey [6.524870790082051]
Deep neural networks (DNNs) have revolutionized artificial intelligence but often lack performance when faced with out-of-distribution (OOD) data.
This survey paper delves into the realm of meta-learning with a focus on its contribution to domain generalization.
arXiv Detail & Related papers (2024-04-03T14:55:17Z) - A Survey on Data Selection for Language Models [148.300726396877]
Data selection methods aim to determine which data points to include in a training dataset.
Deep learning is mostly driven by empirical evidence and experimentation on large-scale data is expensive.
Few organizations have the resources for extensive data selection research.
arXiv Detail & Related papers (2024-02-26T18:54:35Z) - Data Augmentations in Deep Weight Spaces [89.45272760013928]
We introduce a novel augmentation scheme based on the Mixup method.
We evaluate the performance of these techniques on existing benchmarks as well as new benchmarks we generate.
arXiv Detail & Related papers (2023-11-15T10:43:13Z) - Semi-supervised Object Detection: A Survey on Recent Research and
Progress [2.2398477810999817]
Semi-supervised object detection (SSOD) has been paid more and more attentions due to its high research value and practicability.
We present a comprehensive and up-to-date survey on the SSOD approaches from five aspects.
arXiv Detail & Related papers (2023-06-25T02:54:03Z) - A Comprehensive Survey on Source-free Domain Adaptation [69.17622123344327]
The research of Source-Free Domain Adaptation (SFDA) has drawn growing attention in recent years.
We provide a comprehensive survey of recent advances in SFDA and organize them into a unified categorization scheme.
We compare the results of more than 30 representative SFDA methods on three popular classification benchmarks.
arXiv Detail & Related papers (2023-02-23T06:32:09Z) - Deep networks for system identification: a Survey [56.34005280792013]
System identification learns mathematical descriptions of dynamic systems from input-output data.
Main aim of the identified model is to predict new data from previous observations.
We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
arXiv Detail & Related papers (2023-01-30T12:38:31Z) - Data Augmentation techniques in time series domain: A survey and
taxonomy [0.20971479389679332]
Deep neural networks used to work with time series heavily depend on the size and consistency of the datasets used in training.
This work systematically reviews the current state-of-the-art in the area to provide an overview of all available algorithms.
The ultimate aim of this study is to provide a summary of the evolution and performance of areas that produce better results to guide future researchers in this field.
arXiv Detail & Related papers (2022-06-25T17:09:00Z) - Deep Depth Completion: A Survey [26.09557446012222]
We provide a comprehensive literature review that helps readers better grasp the research trends and clearly understand the current advances.
We investigate the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies.
We present a quantitative comparison of model performance on two widely used benchmark datasets, including an indoor and an outdoor dataset.
arXiv Detail & Related papers (2022-05-11T08:24:00Z) - Recent Few-Shot Object Detection Algorithms: A Survey with Performance
Comparison [54.357707168883024]
Few-Shot Object Detection (FSOD) mimics the humans' ability of learning to learn.
FSOD intelligently transfers the learned generic object knowledge from the common heavy-tailed, to the novel long-tailed object classes.
We give an overview of FSOD, including the problem definition, common datasets, and evaluation protocols.
arXiv Detail & Related papers (2022-03-27T04:11:28Z) - Network Comparison Study of Deep Activation Feature Discriminability
with Novel Objects [0.5076419064097732]
State-of-the-art computer visions algorithms have incorporated Deep Neural Networks (DNN) in feature extracting roles, creating Deep Convolutional Activation Features (DeCAF)
This study analyzes the general discriminability of novel object visual appearances encoded into the DeCAF space of six of the leading visual recognition DNN architectures.
arXiv Detail & Related papers (2022-02-08T07:40:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.