Overlapping Word Removal is All You Need: Revisiting Data Imbalance in Hope Speech Detection

• URL: http://arxiv.org/abs/2204.05488v1
• Date: Tue, 12 Apr 2022 02:38:54 GMT
• Title: Overlapping Word Removal is All You Need: Revisiting Data Imbalance in Hope Speech Detection
• Authors: Hariharan RamakrishnaIyer LekshmiAmmal, Manikandan Ravikiran, Gayathri Nisha, Navyasree Balamuralidhar, Adithya Madhusoodanan, Anand Kumar Madasamy, and Bharathi Raja Chakravarthi
• Abstract summary: We introduce focal loss, data augmentation, and pre-processing strategies for hope speech detection. We find that introducing focal loss mitigates the effect of class imbalance and improves overall F1-Macro by 0.11. We also show that overlapping word removal based on pre-processing, though simple, improves F1-Macro by 0.28.
• Score: 2.8341970739919433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hope Speech Detection, a task of recognizing positive expressions, has made significant strides recently. However, much of the current works focus on model development without considering the issue of inherent imbalance in the data. Our work revisits this issue in hope-speech detection by introducing focal loss, data augmentation, and pre-processing strategies. Accordingly, we find that introducing focal loss as part of Multilingual-BERT's (M-BERT) training process mitigates the effect of class imbalance and improves overall F1-Macro by 0.11. At the same time, contextual and back-translation-based word augmentation with M-BERT improves results by 0.10 over baseline despite imbalance. Finally, we show that overlapping word removal based on pre-processing, though simple, improves F1-Macro by 0.28. In due process, we present detailed studies depicting various behaviors of each of these strategies and summarize key findings from our empirical results for those interested in getting the most out of M-BERT for hope speech detection under real-world conditions of data imbalance.

Related papers

• Improving Sampling Methods for Fine-tuning SentenceBERT in Text Streams [49.3179290313959]
  This study explores the efficacy of seven text sampling methods designed to selectively fine-tune language models. We precisely assess the impact of these methods on fine-tuning the SBERT model using four different loss functions. Our findings indicate that Softmax loss and Batch All Triplets loss are particularly effective for text stream classification.
  arXiv  Detail & Related papers  (2024-03-18T23:41:52Z)
• Speech foundation models on intelligibility prediction for hearing-impaired listeners [4.742307809368852]
  Speech foundation models (SFMs) have been benchmarked on many speech processing tasks. We present a systematic evaluation of 10 SFMs on one such application: Speech intelligibility prediction. We propose a simple method that learns a specialized prediction head on top of frozen SFMs to approach the problem.
  arXiv  Detail & Related papers  (2024-01-24T18:26:52Z)
• Automatically measuring speech fluency in people with aphasia: first achievements using read-speech data [55.84746218227712]
  This study aims at assessing the relevance of a signalprocessingalgorithm, initially developed in the field of language acquisition, for the automatic measurement of speech fluency.
  arXiv  Detail & Related papers  (2023-08-09T07:51:40Z)
• UZH_CLyp at SemEval-2023 Task 9: Head-First Fine-Tuning and ChatGPT Data Generation for Cross-Lingual Learning in Tweet Intimacy Prediction [3.1798318618973362]
  This paper describes the submission of UZH_CLyp for the SemEval 2023 Task 9 "Multilingual Tweet Intimacy Analysis" We achieved second-best results in all 10 languages according to the official Pearson's correlation regression evaluation measure.
  arXiv  Detail & Related papers  (2023-03-02T12:18:53Z)
• Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning [7.42741711946564]
  Stuttering is a neuro-developmental speech impairment characterized by uncontrolled utterances and core behaviors. In this paper, we investigate the effectiveness of data augmentation on top of a multi-branched training scheme to tackle data scarcity. In addition, we propose a multi-contextual (MC) StutterNet, which exploits different contexts of the stuttered speech.
  arXiv  Detail & Related papers  (2023-02-21T14:03:47Z)
• Understanding and Mitigating the Uncertainty in Zero-Shot Translation [92.25357943169601]
  We aim to understand and alleviate the off-target issues from the perspective of uncertainty in zero-shot translation. We propose two lightweight and complementary approaches to denoise the training data for model training. Our approaches significantly improve the performance of zero-shot translation over strong MNMT baselines.
  arXiv  Detail & Related papers  (2022-05-20T10:29:46Z)
• Scale-Equivalent Distillation for Semi-Supervised Object Detection [57.59525453301374]
  Recent Semi-Supervised Object Detection (SS-OD) methods are mainly based on self-training, generating hard pseudo-labels by a teacher model on unlabeled data as supervisory signals. We analyze the challenges these methods meet with the empirical experiment results. We introduce a novel approach, Scale-Equivalent Distillation (SED), which is a simple yet effective end-to-end knowledge distillation framework robust to large object size variance and class imbalance.
  arXiv  Detail & Related papers  (2022-03-23T07:33:37Z)
• Deep F-measure Maximization for End-to-End Speech Understanding [52.36496114728355]
  We propose a differentiable approximation to the F-measure and train the network with this objective using standard backpropagation. We perform experiments on two standard fairness datasets, Adult, Communities and Crime, and also on speech-to-intent detection on the ATIS dataset and speech-to-image concept classification on the Speech-COCO dataset. In all four of these tasks, F-measure results in improved micro-F1 scores, with absolute improvements of up to 8% absolute, as compared to models trained with the cross-entropy loss function.
  arXiv  Detail & Related papers  (2020-08-08T03:02:27Z)
• Mutual Information Maximization for Effective Lip Reading [99.11600901751673]
  We propose to introduce the mutual information constraints on both the local feature's level and the global sequence's level. By combining these two advantages together, the proposed method is expected to be both discriminative and robust for effective lip reading.
  arXiv  Detail & Related papers  (2020-03-13T18:47:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.