Classical Sequence Match is a Competitive Few-Shot One-Class Learner

• URL: http://arxiv.org/abs/2209.06394v1
• Date: Wed, 14 Sep 2022 03:21:47 GMT
• Title: Classical Sequence Match is a Competitive Few-Shot One-Class Learner
• Authors: Mengting Hu, Hang Gao, Yinhao Bai, Mingming Liu
• Abstract summary: We investigate the few-shot one-class problem, which actually takes a known sample as a reference to detect whether an unknown instance belongs to the same class. It is shown that with meta-learning, the classical sequence match method, i.e. Compare-Aggregate, significantly outperforms transformer ones.
• Score: 15.598750267663286
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nowadays, transformer-based models gradually become the default choice for artificial intelligence pioneers. The models also show superiority even in the few-shot scenarios. In this paper, we revisit the classical methods and propose a new few-shot alternative. Specifically, we investigate the few-shot one-class problem, which actually takes a known sample as a reference to detect whether an unknown instance belongs to the same class. This problem can be studied from the perspective of sequence match. It is shown that with meta-learning, the classical sequence match method, i.e. Compare-Aggregate, significantly outperforms transformer ones. The classical approach requires much less training cost. Furthermore, we perform an empirical comparison between two kinds of sequence match approaches under simple fine-tuning and meta-learning. Meta-learning causes the transformer models' features to have high-correlation dimensions. The reason is closely related to the number of layers and heads of transformer models. Experimental codes and data are available at https://github.com/hmt2014/FewOne

Related papers

• Generative Multi-modal Models are Good Class-Incremental Learners [51.5648732517187]
  We propose a novel generative multi-modal model (GMM) framework for class-incremental learning. Our approach directly generates labels for images using an adapted generative model. Under the Few-shot CIL setting, we have improved by at least 14% accuracy over all the current state-of-the-art methods with significantly less forgetting.
  arXiv  Detail & Related papers  (2024-03-27T09:21:07Z)
• Class-Incremental Learning: A Survey [84.30083092434938]
  Class-Incremental Learning (CIL) enables the learner to incorporate the knowledge of new classes incrementally. CIL tends to catastrophically forget the characteristics of former ones, and its performance drastically degrades. We provide a rigorous and unified evaluation of 17 methods in benchmark image classification tasks to find out the characteristics of different algorithms.
  arXiv  Detail & Related papers  (2023-02-07T17:59:05Z)
• On the Versatile Uses of Partial Distance Correlation in Deep Learning [47.11577420740119]
  This paper revisits a (less widely known) from statistics, called distance correlation (and its partial variant), designed to evaluate correlation between feature spaces of different dimensions. We describe the steps necessary to carry out its deployment for large scale models. This opens the door to a surprising array of applications ranging from conditioning one deep model w.r.t. another, learning disentangled representations as well as optimizing diverse models that would directly be more robust to adversarial attacks.
  arXiv  Detail & Related papers  (2022-07-20T06:36:11Z)
• What do Toothbrushes do in the Kitchen? How Transformers Think our World is Structured [137.83584233680116]
  We investigate what extent transformer-based language models allow for extracting knowledge about object relations. We show that the models combined with the different similarity measures differ greatly in terms of the amount of knowledge they allow for extracting. Surprisingly, static models perform almost as well as contextualized models -- in some cases even better.
  arXiv  Detail & Related papers  (2022-04-12T10:00:20Z)
• Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning [141.35105358670316]
  We study the difference between a na"ively-trained initial-phase model and the oracle model. We propose Class-wise Decorrelation (CwD) that effectively regularizes representations of each class to scatter more uniformly. Our CwD is simple to implement and easy to plug into existing methods.
  arXiv  Detail & Related papers  (2021-12-09T07:20:32Z)
• A Closer Look at Prototype Classifier for Few-shot Image Classification [28.821731837776593]
  We show that a prototype classifier works equally well without fine-tuning and meta-learning. We derive a novel generalization bound for the prototypical network and show that focusing on the variance of the norm of a feature vector can improve performance.
  arXiv  Detail & Related papers  (2021-10-11T08:28:43Z)
• A Neural Few-Shot Text Classification Reality Check [4.689945062721168]
  Several neural few-shot classification models have emerged, yielding significant progress over time. In this paper, we compare all these models, first adapting those made in the field of image processing to NLP, and second providing them access to transformers. We then test these models equipped with the same transformer-based encoder on the intent detection task, known for having a large number of classes.
  arXiv  Detail & Related papers  (2021-01-28T15:46:14Z)
• One vs Previous and Similar Classes Learning -- A Comparative Study [2.208242292882514]
  This work proposes three learning paradigms which allow trained models to be updated without the need of retraining from scratch. Results show that the proposed paradigms are faster than the baseline at updating, with two of them being faster at training from scratch as well, especially on larger datasets.
  arXiv  Detail & Related papers  (2021-01-05T00:28:38Z)
• Frustratingly Simple Few-Shot Object Detection [98.42824677627581]
  We find that fine-tuning only the last layer of existing detectors on rare classes is crucial to the few-shot object detection task. Such a simple approach outperforms the meta-learning methods by roughly 220 points on current benchmarks.
  arXiv  Detail & Related papers  (2020-03-16T00:29:14Z)
• Unraveling Meta-Learning: Understanding Feature Representations for Few-Shot Tasks [55.66438591090072]
  We develop a better understanding of the underlying mechanics of meta-learning and the difference between models trained using meta-learning and models trained classically. We develop a regularizer which boosts the performance of standard training routines for few-shot classification.
  arXiv  Detail & Related papers  (2020-02-17T03:18:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.