Online Lifelong Generalized Zero-Shot Learning

• URL: http://arxiv.org/abs/2103.10741v2
• Date: Mon, 22 Mar 2021 03:05:03 GMT
• Title: Online Lifelong Generalized Zero-Shot Learning
• Authors: Chandan Gautam, Sethupathy Parameswaran, Ashish Mishra, Suresh Sundaram
• Abstract summary: Methods proposed in the literature for zero-shot learning (ZSL) are typically suitable for offline learning and cannot continually learn from sequential streaming data. This paper proposes a task-free (i.e., task-agnostic) CZSL method, which does not require any task information during continual learning.
• Score: 7.909034037183046
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Methods proposed in the literature for zero-shot learning (ZSL) are typically suitable for offline learning and cannot continually learn from sequential streaming data. The sequential data comes in the form of tasks during training. Recently, a few attempts have been made to handle this issue and develop continual ZSL (CZSL) methods. However, these CZSL methods require clear task-boundary information between the tasks during training, which is not practically possible. This paper proposes a task-free (i.e., task-agnostic) CZSL method, which does not require any task information during continual learning. The proposed task-free CZSL method employs a variational autoencoder (VAE) for performing ZSL. To develop the CZSL method, we combine the concept of experience replay with knowledge distillation and regularization. Here, knowledge distillation is performed using the training sample's dark knowledge, which essentially helps overcome the catastrophic forgetting issue. Further, it is enabled for task-free learning using short-term memory. Finally, a classifier is trained on the synthetic features generated at the latent space of the VAE. Moreover, the experiments are conducted in a challenging and practical ZSL setup, i.e., generalized ZSL (GZSL). These experiments are conducted for two kinds of single-head continual learning settings: (i) mild setting-: task-boundary is known only during training but not during testing; (ii) strict setting-: task-boundary is not known at training, as well as testing. Experimental results on five benchmark datasets exhibit the validity of the approach for CZSL.

Related papers

• Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects [84.6945070729684]
  Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. This article reviews current state-of-the-art SSL methods for time series data.
  arXiv  Detail & Related papers  (2023-06-16T18:23:10Z)
• Improving Open-Set Semi-Supervised Learning with Self-Supervision [13.944469874692459]
  Open-set semi-supervised learning (OSSL) embodies a practical scenario within semi-supervised learning. We propose an OSSL framework that facilitates learning from all unlabeled data through self-supervision. Our method yields state-of-the-art results on many of the evaluated benchmark problems.
  arXiv  Detail & Related papers  (2023-01-24T16:46:37Z)
• A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends [82.64268080902742]
  Self-supervised learning (SSL) aims to learn discriminative features from unlabeled data without relying on human-annotated labels. SSL has garnered significant attention recently, leading to the development of numerous related algorithms. This paper presents a review of diverse SSL methods, encompassing algorithmic aspects, application domains, three key trends, and open research questions.
  arXiv  Detail & Related papers  (2023-01-13T14:41:05Z)
• OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learning [110.40285771431687]
  Semi-supervised learning (SSL) is one of the dominant approaches to address the annotation bottleneck of supervised learning. Recent SSL methods can effectively leverage a large repository of unlabeled data to improve performance while relying on a small set of labeled data. This work introduces OpenLDN that utilizes a pairwise similarity loss to discover novel classes.
  arXiv  Detail & Related papers  (2022-07-05T18:51:05Z)
• DATA: Domain-Aware and Task-Aware Pre-training [94.62676913928831]
  We present DATA, a simple yet effective NAS approach specialized for self-supervised learning (SSL) Our method achieves promising results across a wide range of computation costs on downstream tasks, including image classification, object detection and semantic segmentation.
  arXiv  Detail & Related papers  (2022-03-17T02:38:49Z)
• Learning to Prompt for Continual Learning [34.609384246149325]
  This work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity.
  arXiv  Detail & Related papers  (2021-12-16T06:17:07Z)
• A Strong Baseline for Semi-Supervised Incremental Few-Shot Learning [54.617688468341704]
  Few-shot learning aims to learn models that generalize to novel classes with limited training samples. We propose a novel paradigm containing two parts: (1) a well-designed meta-training algorithm for mitigating ambiguity between base and novel classes caused by unreliable pseudo labels and (2) a model adaptation mechanism to learn discriminative features for novel classes while preserving base knowledge using few labeled and all the unlabeled data.
  arXiv  Detail & Related papers  (2021-10-21T13:25:52Z)
• Self-supervised Regularization for Text Classification [14.824073299035675]
  In many real-world problems, the number of texts for training classification models is limited, which renders these models prone to overfitting. We propose SSL-Reg, a data-dependent regularization approach based on self-supervised learning (SSL) SSL is an unsupervised learning approach which defines auxiliary tasks on input data without using any human-provided labels.
  arXiv  Detail & Related papers  (2021-03-09T05:35:52Z)
• End-to-end Generative Zero-shot Learning via Few-shot Learning [76.9964261884635]
  State-of-the-art approaches to Zero-Shot Learning (ZSL) train generative nets to synthesize examples conditioned on the provided metadata. We introduce an end-to-end generative ZSL framework that uses such an approach as a backbone and feeds its synthesized output to a Few-Shot Learning algorithm.
  arXiv  Detail & Related papers  (2021-02-08T17:35:37Z)
• Generative Replay-based Continual Zero-Shot Learning [7.909034037183046]
  We develop a generative replay-based continual ZSL (GRCZSL) The proposed method endows traditional ZSL to learn from streaming data and acquire new knowledge without forgetting the previous tasks' experience. The proposed GRZSL method is developed for a single-head setting of continual learning, simulating a real-world problem setting.
  arXiv  Detail & Related papers  (2021-01-22T00:03:34Z)
• Generalized Continual Zero-Shot Learning [7.097782028036196]
  zero-shot learning (ZSL) aims to classify unseen classes by transferring the knowledge from seen classes to unseen classes based on the class description. We propose a more general and practical setup for ZSL, where classes arrive sequentially in the form of a task. We use knowledge distillation and storing and replay the few samples from previous tasks using a small episodic memory.
  arXiv  Detail & Related papers  (2020-11-17T08:47:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.