SSMix: Saliency-Based Span Mixup for Text Classification

• URL: http://arxiv.org/abs/2106.08062v1
• Date: Tue, 15 Jun 2021 11:40:23 GMT
• Title: SSMix: Saliency-Based Span Mixup for Text Classification
• Authors: Soyoung Yoon, Gyuwan Kim, Kyumin Park
• Abstract summary: We propose SSMix, a novel mixup method where the operation is performed on input text rather than on hidden vectors. SSMix synthesizes a sentence while preserving the locality of two original texts by span-based mixing. We empirically validate that our method outperforms hidden-level mixup methods on a wide range of text classification benchmarks.
• Score: 2.4493299476776778
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data augmentation with mixup has shown to be effective on various computer vision tasks. Despite its great success, there has been a hurdle to apply mixup to NLP tasks since text consists of discrete tokens with variable length. In this work, we propose SSMix, a novel mixup method where the operation is performed on input text rather than on hidden vectors like previous approaches. SSMix synthesizes a sentence while preserving the locality of two original texts by span-based mixing and keeping more tokens related to the prediction relying on saliency information. With extensive experiments, we empirically validate that our method outperforms hidden-level mixup methods on a wide range of text classification benchmarks, including textual entailment, sentiment classification, and question-type classification. Our code is available at https://github.com/clovaai/ssmix.

Related papers

• Elevating Code-mixed Text Handling through Auditory Information of Words [24.53638976212391]
  We propose an effective approach for creating language models for handling code-mixed textual data using auditory information of words from SOUNDEX. Our approach includes a pre-training step based on masked-language-modelling, which includes SOUNDEX representations (SAMLM) and a new method of providing input data to the pre-trained model.
  arXiv  Detail & Related papers  (2023-10-27T14:03:30Z)
• HIT-SCIR at MMNLU-22: Consistency Regularization for Multilingual Spoken Language Understanding [56.756090143062536]
  We propose to use consistency regularization based on a hybrid data augmentation strategy. We conduct experiments on the MASSIVE dataset under both full-dataset and zero-shot settings. Our proposed method improves the performance on both intent detection and slot filling tasks.
  arXiv  Detail & Related papers  (2023-01-05T11:21:15Z)
• SelfMix: Robust Learning Against Textual Label Noise with Self-Mixup Training [15.877178854064708]
  SelfMix is a simple yet effective method to handle label noise in text classification tasks. Our method utilizes the dropout mechanism on a single model to reduce the confirmation bias in self-training.
  arXiv  Detail & Related papers  (2022-10-10T09:46:40Z)
• DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification [56.817386699291305]
  This paper proposes a simple yet effective data augmentation approach termed DoubleMix. DoubleMix first generates several perturbed samples for each training data. It then uses the perturbed data and original data to carry out a two-step in the hidden space of neural models.
  arXiv  Detail & Related papers  (2022-09-12T15:01:04Z)
• SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data [124.95585891086894]
  Proposal is called Semantically Proportional Mixing (SnapMix) It exploits class activation map (CAM) to lessen the label noise in augmenting fine-grained data. Our method consistently outperforms existing mixed-based approaches.
  arXiv  Detail & Related papers  (2020-12-09T03:37:30Z)
• Sequence-Level Mixed Sample Data Augmentation [119.94667752029143]
  This work proposes a simple data augmentation approach to encourage compositional behavior in neural models for sequence-to-sequence problems. Our approach, SeqMix, creates new synthetic examples by softly combining input/output sequences from the training set.
  arXiv  Detail & Related papers  (2020-11-18T02:18:04Z)
• IIT Gandhinagar at SemEval-2020 Task 9: Code-Mixed Sentiment Classification Using Candidate Sentence Generation and Selection [1.2301855531996841]
  Code-mixing adds to the challenge of analyzing the sentiment of the text due to the non-standard writing style. We present a candidate sentence generation and selection based approach on top of the Bi-LSTM based neural classifier. The proposed approach shows an improvement in the system performance as compared to the Bi-LSTM based neural classifier.
  arXiv  Detail & Related papers  (2020-06-25T14:59:47Z)
• MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification [68.15015032551214]
  MixText is a semi-supervised learning method for text classification. TMix creates a large amount of augmented training samples by interpolating text in hidden space. We leverage recent advances in data augmentation to guess low-entropy labels for unlabeled data.
  arXiv  Detail & Related papers  (2020-04-25T21:37:36Z)
• Learning to Select Bi-Aspect Information for Document-Scale Text Content Manipulation [50.01708049531156]
  We focus on a new practical task, document-scale text content manipulation, which is the opposite of text style transfer. In detail, the input is a set of structured records and a reference text for describing another recordset. The output is a summary that accurately describes the partial content in the source recordset with the same writing style of the reference.
  arXiv  Detail & Related papers  (2020-02-24T12:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.