Statistical Measures For Defining Curriculum Scoring Function

• URL: http://arxiv.org/abs/2103.00147v1
• Date: Sat, 27 Feb 2021 07:25:49 GMT
• Title: Statistical Measures For Defining Curriculum Scoring Function
• Authors: Vinu Sankar Sadasivan, Anirban Dasgupta
• Abstract summary: We show improvements in performance with convolutional and fully-connected neural networks on real image datasets. Motivated by our insights from implicit curriculum ordering, we introduce a simple curriculum learning strategy. We also propose and study the performance of a dynamic curriculum learning algorithm.
• Score: 5.328970912536596
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Curriculum learning is a training strategy that sorts the training examples by some measure of their difficulty and gradually exposes them to the learner to improve the network performance. In this work, we propose two novel curriculum learning algorithms, and empirically show their improvements in performance with convolutional and fully-connected neural networks on multiple real image datasets. Motivated by our insights from implicit curriculum ordering, we introduce a simple curriculum learning strategy that uses statistical measures such as standard deviation and entropy values to score the difficulty of data points for real image classification tasks. We also propose and study the performance of a dynamic curriculum learning algorithm. Our dynamic curriculum algorithm tries to reduce the distance between the network weight and an optimal weight at any training step by greedily sampling examples with gradients that are directed towards the optimal weight. Further, we also use our algorithms to discuss why curriculum learning is helpful.

Related papers

• EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training [79.96741042766524]
  We reformulate the training curriculum as a soft-selection function. We show that exposing the contents of natural images can be readily achieved by the intensity of data augmentation. The resulting method, EfficientTrain++, is simple, general, yet surprisingly effective.
  arXiv  Detail & Related papers  (2024-05-14T17:00:43Z)
• Dynamic Task and Weight Prioritization Curriculum Learning for Multimodal Imagery [0.5439020425819]
  This paper explores post-disaster analytics using multimodal deep learning models trained with curriculum learning method. Curriculum learning emulates the progressive learning sequence in human education by training deep learning models on increasingly complex data.
  arXiv  Detail & Related papers  (2023-10-29T18:46:33Z)
• Curriculum Learning for Graph Neural Networks: A Multiview Competence-based Approach [12.335698325757491]
  We propose a new perspective on curriculum learning by introducing a novel approach that builds on graph complexity formalisms. The proposed solution advances existing research in curriculum learning for graph neural networks with the ability to incorporate a fine-grained spectrum of graph difficulty criteria.
  arXiv  Detail & Related papers  (2023-07-17T21:33:35Z)
• Online Network Source Optimization with Graph-Kernel MAB [62.6067511147939]
  We propose Grab-UCB, a graph- kernel multi-arms bandit algorithm to learn online the optimal source placement in large scale networks. We describe the network processes with an adaptive graph dictionary model, which typically leads to sparse spectral representations. We derive the performance guarantees that depend on network parameters, which further influence the learning curve of the sequential decision strategy.
  arXiv  Detail & Related papers  (2023-07-07T15:03:42Z)
• ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
  We introduce Action-Aware Embodied Learning for Perception (ALP) ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. We show that ALP outperforms existing baselines in several downstream perception tasks.
  arXiv  Detail & Related papers  (2023-06-16T21:51:04Z)
• EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones [80.662250618795]
  This paper presents a new curriculum learning approach for the efficient training of visual backbones (e.g., vision Transformers) As an off-the-shelf method, it reduces the wall-time training cost of a wide variety of popular models by >1.5x on ImageNet-1K/22K without sacrificing accuracy.
  arXiv  Detail & Related papers  (2022-11-17T17:38:55Z)
• Budget-aware Few-shot Learning via Graph Convolutional Network [56.41899553037247]
  This paper tackles the problem of few-shot learning, which aims to learn new visual concepts from a few examples. A common problem setting in few-shot classification assumes random sampling strategy in acquiring data labels. We introduce a new budget-aware few-shot learning problem that aims to learn novel object categories.
  arXiv  Detail & Related papers  (2022-01-07T02:46:35Z)
• Curriculum Meta-Learning for Few-shot Classification [1.5039745292757671]
  We propose an adaptation of the curriculum training framework, applicable to state-of-the-art meta learning techniques for few-shot classification. Our experiments with the MAML algorithm on two few-shot image classification tasks show significant gains with the curriculum training framework.
  arXiv  Detail & Related papers  (2021-12-06T10:29:23Z)
• Spatial Transformer Networks for Curriculum Learning [6.107943372244105]
  We take inspiration from Spatial Transformer Networks (STNs) in order to form an easy-to-hard curriculum. We perform various experiments on cluttered MNIST and Fashion-MNIST datasets.
  arXiv  Detail & Related papers  (2021-08-22T12:04:55Z)
• Curriculum Learning: A Survey [65.31516318260759]
  Curriculum learning strategies have been successfully employed in all areas of machine learning. We construct a taxonomy of curriculum learning approaches by hand, considering various classification criteria. We build a hierarchical tree of curriculum learning methods using an agglomerative clustering algorithm.
  arXiv  Detail & Related papers  (2021-01-25T20:08:32Z)
• Curriculum Learning with Diversity for Supervised Computer Vision Tasks [1.5229257192293197]
  We introduce a novel curriculum sampling strategy which takes into consideration the diversity of the training data together with the difficulty of the inputs. We prove that our strategy is very efficient for unbalanced data sets, leading to faster convergence and more accurate results.
  arXiv  Detail & Related papers  (2020-09-22T15:32:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.