SAT: Improving Semi-Supervised Text Classification with Simple Instance-Adaptive Self-Training

• URL: http://arxiv.org/abs/2210.12653v1
• Date: Sun, 23 Oct 2022 08:19:58 GMT
• Title: SAT: Improving Semi-Supervised Text Classification with Simple Instance-Adaptive Self-Training
• Authors: Hui Chen, Wei Han, Soujanya Poria
• Abstract summary: This work presents a Simple instance-Adaptive self-Training method (SAT) for semi-supervised text classification. SAT first generates two augmented views for each unlabeled data and then trains a meta-learner to automatically identify the relative strength of augmentations.
• Score: 19.879452265836917
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-training methods have been explored in recent years and have exhibited great performance in improving semi-supervised learning. This work presents a Simple instance-Adaptive self-Training method (SAT) for semi-supervised text classification. SAT first generates two augmented views for each unlabeled data and then trains a meta-learner to automatically identify the relative strength of augmentations based on the similarity between the original view and the augmented views. The weakly-augmented view is fed to the model to produce a pseudo-label and the strongly-augmented view is used to train the model to predict the same pseudo-label. We conducted extensive experiments and analyses on three text classification datasets and found that with varying sizes of labeled training data, SAT consistently shows competitive performance compared to existing semi-supervised learning methods. Our code can be found at \url{https://github.com/declare-lab/SAT.git}.

Related papers

• Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning [50.26965628047682]
  Adapting pre-trained models to open classes is a challenging problem in machine learning. In this paper, we consider combining the advantages of both and come up with a test-time prompt tuning approach. Our proposed method outperforms all comparison methods on average considering both base and new classes.
  arXiv  Detail & Related papers  (2024-08-29T12:34:01Z)
• Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models [3.546617486894182]
  We introduce HAST, a new and effective self-training strategy, which is evaluated on four text classification benchmarks. Results show that it outperforms the reproduced self-training approaches and reaches classification results comparable to previous experiments for three out of four datasets.
  arXiv  Detail & Related papers  (2024-06-13T15:06:11Z)
• Meta Co-Training: Two Views are Better than One [4.050257210426548]
  We present Meta Co-Training which is an extension of the successful Meta Pseudo Labels approach to two views. Our method achieves new state-of-the-art performance on ImageNet-10% with very few training resources.
  arXiv  Detail & Related papers  (2023-11-29T21:11:58Z)
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond [114.95815360508395]
  This paper presents one-bit supervision, a novel setting of learning with fewer labels, for image classification. We propose a multi-stage training paradigm and incorporate negative label suppression into an off-the-shelf semi-supervised learning algorithm. In multiple benchmarks, the learning efficiency of the proposed approach surpasses that using full-bit, semi-supervised supervision.
  arXiv  Detail & Related papers  (2023-11-26T07:39:00Z)
• M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios [103.6153593636399]
  We propose a vision-language prompt tuning method with mitigated label bias (M-Tuning) It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more, and thus prompts are tuned in a simulated open-set scenario. Our method achieves the best performance on datasets with various scales, and extensive ablation studies also validate its effectiveness.
  arXiv  Detail & Related papers  (2023-03-09T09:05:47Z)
• Dense FixMatch: a simple semi-supervised learning method for pixel-wise prediction tasks [68.36996813591425]
  We propose Dense FixMatch, a simple method for online semi-supervised learning of dense and structured prediction tasks. We enable the application of FixMatch in semi-supervised learning problems beyond image classification by adding a matching operation on the pseudo-labels. Dense FixMatch significantly improves results compared to supervised learning using only labeled data, approaching its performance with 1/4 of the labeled samples.
  arXiv  Detail & Related papers  (2022-10-18T15:02:51Z)
• DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification [56.817386699291305]
  This paper proposes a simple yet effective data augmentation approach termed DoubleMix. DoubleMix first generates several perturbed samples for each training data. It then uses the perturbed data and original data to carry out a two-step in the hidden space of neural models.
  arXiv  Detail & Related papers  (2022-09-12T15:01:04Z)
• Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels [26.542718087103665]
  SemiMatch is a semi-supervised solution for establishing dense correspondences across semantically similar images. Our framework generates the pseudo-labels using the model's prediction itself between source and weakly-augmented target, and uses pseudo-labels to learn the model again between source and strongly-augmented target. In experiments, SemiMatch achieves state-of-the-art performance on various benchmarks, especially on PF-Willow by a large margin.
  arXiv  Detail & Related papers  (2022-03-30T03:52:50Z)
• SimMatch: Semi-supervised Learning with Similarity Matching [43.61802702362675]
  SimMatch is a new semi-supervised learning framework that considers semantic similarity and instance similarity. With 400 epochs of training, SimMatch achieves 67.2%, and 74.4% Top-1 Accuracy with 1% and 10% labeled examples on ImageNet.
  arXiv  Detail & Related papers  (2022-03-14T08:08:48Z)
• SLADE: A Self-Training Framework For Distance Metric Learning [75.54078592084217]
  We present a self-training framework, SLADE, to improve retrieval performance by leveraging additional unlabeled data. We first train a teacher model on the labeled data and use it to generate pseudo labels for the unlabeled data. We then train a student model on both labels and pseudo labels to generate final feature embeddings.
  arXiv  Detail & Related papers  (2020-11-20T08:26:10Z)
• Semi-supervised dictionary learning with graph regularization and active points [0.19947949439280027]
  We propose a new semi-supervised dictionary learning method based on two pillars. On one hand, we enforce manifold structure preservation from the original data into sparse code space using Locally Linear Embedding. On the other hand, we train a semi-supervised classifier in sparse code space.
  arXiv  Detail & Related papers  (2020-09-13T09:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.