Semantics-driven Attentive Few-shot Learning over Clean and Noisy Samples

• URL: http://arxiv.org/abs/2201.03043v1
• Date: Sun, 9 Jan 2022 16:16:23 GMT
• Title: Semantics-driven Attentive Few-shot Learning over Clean and Noisy Samples
• Authors: Orhun Bu\u{g}ra Baran and Ramazan G\"okberk Cinbi\c{s}
• Abstract summary: We aim to train meta-learner models that can leverage prior semantic knowledge about novel classes to guide the classifier synthesis process. In particular, we propose semantically-conditioned feature attention and sample attention mechanisms that estimate the importance of representation dimensions and training instances.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Over the last couple of years few-shot learning (FSL) has attracted great attention towards minimizing the dependency on labeled training examples. An inherent difficulty in FSL is the handling of ambiguities resulting from having too few training samples per class. To tackle this fundamental challenge in FSL, we aim to train meta-learner models that can leverage prior semantic knowledge about novel classes to guide the classifier synthesis process. In particular, we propose semantically-conditioned feature attention and sample attention mechanisms that estimate the importance of representation dimensions and training instances. We also study the problem of sample noise in FSL, towards the utilization of meta-learners in more realistic and imperfect settings. Our experimental results demonstrate the effectiveness of the proposed semantic FSL model with and without sample noise.

Related papers

• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning [4.137391543972184]
  Semi-supervised learning (SSL) has witnessed remarkable progress, resulting in numerous method variations. In this paper, we present a novel SSL approach named FineSSL that significantly addresses this limitation by adapting pre-trained foundation models. We demonstrate that FineSSL sets a new state of the art for SSL on multiple benchmark datasets, reduces the training cost by over six times, and can seamlessly integrate various fine-tuning and modern SSL algorithms.
  arXiv  Detail & Related papers  (2024-05-20T03:33:12Z)
• FSL-Rectifier: Rectify Outliers in Few-Shot Learning via Test-Time Augmentation [0.1843404256219181]
  Few-shot-learning (FSL) commonly requires a model to identify images (queries) that belong to classes unseen during training. In this work, we generate additional test-class samples through combining them with suitable train-class samples. We experimentally and theoretically demonstrate the effectiveness of our method, via which we can achieve around 4% test accuracy improvement.
  arXiv  Detail & Related papers  (2024-02-28T12:37:30Z)
• Combating Label Noise With A General Surrogate Model For Sample Selection [84.61367781175984]
  We propose to leverage the vision-language surrogate model CLIP to filter noisy samples automatically. We validate the effectiveness of our proposed method on both real-world and synthetic noisy datasets.
  arXiv  Detail & Related papers  (2023-10-16T14:43:27Z)
• Adversarial Robustness of Prompt-based Few-Shot Learning for Natural Language Understanding [23.458843951563978]
  State-of-the-art few-shot learning methods leverage prompt-based fine-tuning to obtain remarkable results for natural language understanding (NLU) tasks. We conduct an extensive study of several state-of-the-art FSL methods to assess their robustness to adversarial perturbations.
  arXiv  Detail & Related papers  (2023-06-19T17:01:13Z)
• Constrained Few-Shot Learning: Human-Like Low Sample Complexity Learning and Non-Episodic Text Classification [11.35732215154172]
  Few-shot learning is an emergent paradigm of learning that attempts to learn to reason with low sample complexity. We propose a method for CFSL leveraging Cat2Vec using a novel categorical contrastive loss inspired by cognitive theories.
  arXiv  Detail & Related papers  (2022-08-17T06:05:41Z)
• Pseudo-Labeling Based Practical Semi-Supervised Meta-Training for Few-Shot Learning [125.98370880855579]
  We propose a simple and effective meta-training framework, called pseudo-labeling based meta-learning (PLML) Firstly, we train a classifier via common semi-supervised learning (SSL) and use it to obtain the pseudo-labels of unlabeled data. We build few-shot tasks from labeled and pseudo-labeled data and design a novel finetuning method with feature smoothing and noise suppression.
  arXiv  Detail & Related papers  (2022-07-14T10:53:53Z)
• A Strong Baseline for Semi-Supervised Incremental Few-Shot Learning [35.035361304569186]
  Few-shot learning aims to learn models that generalize to novel classes with limited training samples. We propose a novel paradigm containing two parts: (1) a well-designed meta-training algorithm for mitigating ambiguity between base and novel classes caused by unreliable pseudo labels and (2) a model adaptation mechanism to learn discriminative features for novel classes while preserving base knowledge using few labeled and all the unlabeled data.
  arXiv  Detail & Related papers  (2021-10-21T13:25:52Z)
• Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning [96.75889543560497]
  In many real-world problems, collecting a large number of labeled samples is infeasible. Few-shot learning is the dominant approach to address this issue, where the objective is to quickly adapt to novel categories in presence of a limited number of samples. We propose a novel training mechanism that simultaneously enforces equivariance and invariance to a general set of geometric transformations.
  arXiv  Detail & Related papers  (2021-03-01T21:14:33Z)
• On Data-Augmentation and Consistency-Based Semi-Supervised Learning [77.57285768500225]
  Recently proposed consistency-based Semi-Supervised Learning (SSL) methods have advanced the state of the art in several SSL tasks. Despite these advances, the understanding of these methods is still relatively limited.
  arXiv  Detail & Related papers  (2021-01-18T10:12:31Z)
• Boosting Few-Shot Learning With Adaptive Margin Loss [109.03665126222619]
  This paper proposes an adaptive margin principle to improve the generalization ability of metric-based meta-learning approaches for few-shot learning problems. Extensive experiments demonstrate that the proposed method can boost the performance of current metric-based meta-learning approaches.
  arXiv  Detail & Related papers  (2020-05-28T07:58:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.