Analyzing Curriculum Learning for Sentiment Analysis along Task Difficulty, Pacing and Visualization Axes

• URL: http://arxiv.org/abs/2102.09990v1
• Date: Fri, 19 Feb 2021 15:42:14 GMT
• Title: Analyzing Curriculum Learning for Sentiment Analysis along Task Difficulty, Pacing and Visualization Axes
• Authors: Anvesh Rao Vijjini, Kaveri Anuranjana, Radhika Mamidi
• Abstract summary: We analyze curriculum learning in sentiment analysis along multiple axes. We find that curriculum learning works best for difficult tasks and may even lead to a decrement in performance for tasks that have higher performance without curriculum learning.
• Score: 7.817598216459955
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: While Curriculum learning (CL) has recently gained traction in Natural language Processing Tasks, it still isn't being analyzed adequately. Previous works only show their effectiveness but fail short to fully explain and interpret the internal workings. In this paper, we analyze curriculum learning in sentiment analysis along multiple axes. Some of these axes have been proposed by earlier works that need deeper study. Such analysis requires understanding where curriculum learning works and where it doesn't. Our axes of analysis include Task difficulty on CL, comparing CL pacing techniques, and qualitative analysis by visualizing the movement of attention scores in the model as curriculum phases progress. We find that curriculum learning works best for difficult tasks and may even lead to a decrement in performance for tasks that have higher performance without curriculum learning. Within curriculum pacing, we see that One-Pass curriculum strategies suffer from catastrophic forgetting and attention movement visualization shows that curriculum learning breaks down the main task into easier sub-tasks which the model solves easily.

Related papers

• CLIMB: Curriculum Learning for Infant-inspired Model Building [6.4766496232839685]
  We describe our team's contribution to the STRICT-SMALL track of the BabyLM Challenge. The challenge requires training a language model from scratch using only a relatively small training dataset of ten million words. We experiment with three variants of cognitively-motivated curriculum learning and analyze their effect on the performance of the model.
  arXiv  Detail & Related papers  (2023-11-15T11:48:16Z)
• Dynamic Task and Weight Prioritization Curriculum Learning for Multimodal Imagery [0.5439020425819]
  This paper explores post-disaster analytics using multimodal deep learning models trained with curriculum learning method. Curriculum learning emulates the progressive learning sequence in human education by training deep learning models on increasingly complex data.
  arXiv  Detail & Related papers  (2023-10-29T18:46:33Z)
• Unsupervised Task Graph Generation from Instructional Video Transcripts [53.54435048879365]
  We consider a setting where text transcripts of instructional videos performing a real-world activity are provided. The goal is to identify the key steps relevant to the task as well as the dependency relationship between these key steps. We propose a novel task graph generation approach that combines the reasoning capabilities of instruction-tuned language models along with clustering and ranking components.
  arXiv  Detail & Related papers  (2023-02-17T22:50:08Z)
• What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks? [59.51457877578138]
  We study contrastive learning on the wearable-based activity recognition task. This paper presents an open-source PyTorch library textttCL-HAR, which can serve as a practical tool for researchers.
  arXiv  Detail & Related papers  (2022-02-12T06:10:15Z)
• An Analytical Theory of Curriculum Learning in Teacher-Student Networks [10.303947049948107]
  In humans and animals, curriculum learning is critical to rapid learning and effective pedagogy. In machine learning, curricula are not widely used and empirically often yield only moderate benefits.
  arXiv  Detail & Related papers  (2021-06-15T11:48:52Z)
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study [75.42182503265056]
  Multi-Task Learning has emerged as a methodology in which multiple tasks are jointly learned by a shared learning algorithm. We deal with heterogeneous MTL, simultaneously addressing detection, classification & regression problems. We build FaceBehaviorNet, the first framework for large-scale face analysis, by jointly learning all facial behavior tasks.
  arXiv  Detail & Related papers  (2021-05-08T22:26:52Z)
• Statistical Measures For Defining Curriculum Scoring Function [5.328970912536596]
  We show improvements in performance with convolutional and fully-connected neural networks on real image datasets. Motivated by our insights from implicit curriculum ordering, we introduce a simple curriculum learning strategy. We also propose and study the performance of a dynamic curriculum learning algorithm.
  arXiv  Detail & Related papers  (2021-02-27T07:25:49Z)
• Curriculum Learning: A Survey [65.31516318260759]
  Curriculum learning strategies have been successfully employed in all areas of machine learning. We construct a taxonomy of curriculum learning approaches by hand, considering various classification criteria. We build a hierarchical tree of curriculum learning methods using an agglomerative clustering algorithm.
  arXiv  Detail & Related papers  (2021-01-25T20:08:32Z)
• When Do Curricula Work? [26.072472732516335]
  ordered learning has been suggested as improvements to the standard i.i.d. training. We conduct experiments over thousands of orderings spanning three kinds of learning: curriculum, anti-curriculum, and random-curriculum. We find that curricula have only marginal benefits, and that randomly ordered samples perform as well or better than curricula and anti-curricula.
  arXiv  Detail & Related papers  (2020-12-05T19:41:30Z)
• Pre-training Text Representations as Meta Learning [113.3361289756749]
  We introduce a learning algorithm which directly optimize model's ability to learn text representations for effective learning of downstream tasks. We show that there is an intrinsic connection between multi-task pre-training and model-agnostic meta-learning with a sequence of meta-train steps.
  arXiv  Detail & Related papers  (2020-04-12T09:05:47Z)
• Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey [53.73359052511171]
  Reinforcement learning (RL) is a popular paradigm for addressing sequential decision tasks in which the agent has only limited environmental feedback. We present a framework for curriculum learning (CL) in RL, and use it to survey and classify existing CL methods in terms of their assumptions, capabilities, and goals.
  arXiv  Detail & Related papers  (2020-03-10T20:41:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.