VIPriors 1: Visual Inductive Priors for Data-Efficient Deep Learning Challenges

• URL: http://arxiv.org/abs/2103.03768v1
• Date: Fri, 5 Mar 2021 15:58:17 GMT
• Title: VIPriors 1: Visual Inductive Priors for Data-Efficient Deep Learning Challenges
• Authors: Robert-Jan Bruintjes, Attila Lengyel, Marcos Baptista Rios, Osman Semih Kayhan, Jan van Gemert
• Abstract summary: We offer four data-impaired challenges, where models are trained from scratch, and we reduce the number of training samples to a fraction of the full set. To encourage data efficient solutions, we prohibited the use of pre-trained models and other transfer learning techniques.
• Score: 8.50468505606714
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present the first edition of "VIPriors: Visual Inductive Priors for Data-Efficient Deep Learning" challenges. We offer four data-impaired challenges, where models are trained from scratch, and we reduce the number of training samples to a fraction of the full set. Furthermore, to encourage data efficient solutions, we prohibited the use of pre-trained models and other transfer learning techniques. The majority of top ranking solutions make heavy use of data augmentation, model ensembling, and novel and efficient network architectures to achieve significant performance increases compared to the provided baselines.

Related papers

• VIPriors 4: Visual Inductive Priors for Data-Efficient Deep Learning Challenges [12.615348941903594]
  Fourth edition of the "VIPriors: Visual Inductive Priors for Data-Efficient Deep Learning" workshop features two data-impaired challenges. These challenges address the problem of training deep learning models for computer vision tasks with limited data. We aim to stimulate the development of novel approaches that incorporate inductive biases to improve the data efficiency of deep learning models.
  arXiv  Detail & Related papers  (2024-06-26T08:50:51Z)
• Augment Before Copy-Paste: Data and Memory Efficiency-Oriented Instance Segmentation Framework for Sport-scenes [7.765333471208582]
  In Visual Inductive Priors challenge (VIPriors2023), participants must train a model capable of precisely locating individuals on a basketball court. We propose memory effIciency inStance framework based on visual inductive prior flow propagation. Experiments demonstrate our model promising performance even under limited data and memory constraints.
  arXiv  Detail & Related papers  (2024-03-18T08:44:40Z)
• Less is More: High-value Data Selection for Visual Instruction Tuning [127.38740043393527]
  We propose a high-value data selection approach TIVE, to eliminate redundancy within the visual instruction data and reduce the training cost. Our approach using only about 15% data can achieve comparable average performance to the full-data fine-tuned model across eight benchmarks.
  arXiv  Detail & Related papers  (2024-03-14T16:47:25Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• A Pre-trained Data Deduplication Model based on Active Learning [13.495903601474819]
  "dirty data" problems can significantly limit the effective application of big data. We propose a pre-trained deduplication model based on active learning. Our proposed model outperforms previous state-of-the-art (SOTA) for deduplicated data identification.
  arXiv  Detail & Related papers  (2023-07-31T03:56:46Z)
• VIPriors 3: Visual Inductive Priors for Data-Efficient Deep Learning Challenges [13.085098213230568]
  Third edition of the "VIPriors: Visual Inductive Priors for Data-Efficient Deep Learning" workshop featured four data-impaired challenges. Challenges focused on addressing the limitations of data availability in training deep learning models for computer vision tasks.
  arXiv  Detail & Related papers  (2023-05-31T09:31:54Z)
• Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizes [100.69714600180895]
  offline Q-learning algorithms exhibit strong performance that scales with model capacity. We train a single policy on 40 games with near-human performance using up-to 80 million parameter networks. Compared to return-conditioned supervised approaches, offline Q-learning scales similarly with model capacity and has better performance, especially when the dataset is suboptimal.
  arXiv  Detail & Related papers  (2022-11-28T08:56:42Z)
• Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferability [53.27240222619834]
  Knowledge Distillation as Efficient Pre-training aims to efficiently transfer the learned feature representation from pre-trained models to new student models for future downstream tasks. Our method performs comparably with supervised pre-training counterparts in 3 downstream tasks and 9 downstream datasets requiring 10x less data and 5x less pre-training time.
  arXiv  Detail & Related papers  (2022-03-10T06:23:41Z)
• VIPriors 2: Visual Inductive Priors for Data-Efficient Deep Learning Challenges [13.085098213230568]
  Second edition of "VIPriors: Visual Inductive Priors for Data-Efficient Deep Learning" challenges. Models are trained from scratch on a reduced number of training samples for various key computer vision tasks. Results: The provided baselines are outperformed by a large margin in all five challenges.
  arXiv  Detail & Related papers  (2022-01-21T10:20:52Z)
• Efficient Conditional Pre-training for Transfer Learning [71.01129334495553]
  We propose efficient filtering methods to select relevant subsets from the pre-training dataset. We validate our techniques by pre-training on ImageNet in both the unsupervised and supervised settings. We improve standard ImageNet pre-training by 1-3% by tuning available models on our subsets and pre-training on a dataset filtered from a larger scale dataset.
  arXiv  Detail & Related papers  (2020-11-20T06:16:15Z)
• Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
  We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training. We experimentally verify that the new dataset can significantly improve the ability of the learned FER model. To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
  arXiv  Detail & Related papers  (2020-05-18T09:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.