Label-aware Hard Negative Sampling Strategies with Momentum Contrastive Learning for Implicit Hate Speech Detection

• URL: http://arxiv.org/abs/2406.07886v1
• Date: Wed, 12 Jun 2024 05:24:58 GMT
• Title: Label-aware Hard Negative Sampling Strategies with Momentum Contrastive Learning for Implicit Hate Speech Detection
• Authors: Jaehoon Kim, Seungwan Jin, Sohyun Park, Someen Park, Kyungsik Han,
• Abstract summary: We find that contrastive learning based on randomly sampled batch data does not encourage the model to learn hard negative samples. We propose Label-aware Hard Negative sampling strategies (LAHN) that encourage the model to learn detailed features from hard negative samples. LAHN outperforms the existing models for implicit hate speech detection both in- and cross-datasets.
• Score: 10.436987814180544
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Detecting implicit hate speech that is not directly hateful remains a challenge. Recent research has attempted to detect implicit hate speech by applying contrastive learning to pre-trained language models such as BERT and RoBERTa, but the proposed models still do not have a significant advantage over cross-entropy loss-based learning. We found that contrastive learning based on randomly sampled batch data does not encourage the model to learn hard negative samples. In this work, we propose Label-aware Hard Negative sampling strategies (LAHN) that encourage the model to learn detailed features from hard negative samples, instead of naive negative samples in random batch, using momentum-integrated contrastive learning. LAHN outperforms the existing models for implicit hate speech detection both in- and cross-datasets. The code is available at https://github.com/Hanyang-HCC-Lab/LAHN

Related papers

• Generating Enhanced Negatives for Training Language-Based Object Detectors [86.1914216335631]
  We propose to leverage the vast knowledge built into modern generative models to automatically build negatives that are more relevant to the original data. Specifically, we use large-language-models to generate negative text descriptions, and text-to-image diffusion models to also generate corresponding negative images. Our experimental analysis confirms the relevance of the generated negative data, and its use in language-based detectors improves performance on two complex benchmarks.
  arXiv  Detail & Related papers  (2023-12-29T23:04:00Z)
• Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination [62.18768931714238]
  We propose a novel False Negative Elimination (FNE) strategy to select negatives via sampling. The results demonstrate the superiority of our proposed false negative elimination strategy.
  arXiv  Detail & Related papers  (2023-08-08T16:31:43Z)
• Robust Positive-Unlabeled Learning via Noise Negative Sample Self-correction [48.929877651182885]
  Learning from positive and unlabeled data is known as positive-unlabeled (PU) learning in literature. We propose a new robust PU learning method with a training strategy motivated by the nature of human learning.
  arXiv  Detail & Related papers  (2023-08-01T04:34:52Z)
• Clustering-Aware Negative Sampling for Unsupervised Sentence Representation [24.15096466098421]
  ClusterNS is a novel method that incorporates cluster information into contrastive learning for unsupervised sentence representation learning. We apply a modified K-means clustering algorithm to supply hard negatives and recognize in-batch false negatives during training.
  arXiv  Detail & Related papers  (2023-05-17T02:06:47Z)
• Language Model Pre-training on True Negatives [109.73819321246062]
  Discriminative pre-trained language models (PLMs) learn to predict original texts from intentionally corrupted ones. Existing PLMs simply treat all corrupted texts as equal negative without any examination. We design enhanced pre-training methods to counteract false negative predictions and encourage pre-training language models on true negatives.
  arXiv  Detail & Related papers  (2022-12-01T12:24:19Z)
• Unrestricted Adversarial Samples Based on Non-semantic Feature Clusters Substitution [1.8782750537161608]
  We introduce "unrestricted" perturbations that create adversarial samples by using spurious relations learned by model training. Specifically, we find feature clusters in non-semantic features that are strongly correlated with model judgment results. We create adversarial samples by using them to replace the corresponding feature clusters in the target image.
  arXiv  Detail & Related papers  (2022-08-31T07:42:36Z)
• Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding [14.295787044482136]
  We present a momentum contrastive learning model with negative sample queue for sentence embedding, namely MoCoSE. We define a maximum traceable distance metric, through which we learn to what extent the text contrastive learning benefits from the historical information of negative samples. Our experiments find that the best results are obtained when the maximum traceable distance is at a certain range, demonstrating that there is an optimal range of historical information for a negative sample queue.
  arXiv  Detail & Related papers  (2022-02-26T08:29:25Z)
• Contrastive Learning with Hard Negative Samples [80.12117639845678]
  We develop a new family of unsupervised sampling methods for selecting hard negative samples. A limiting case of this sampling results in a representation that tightly clusters each class, and pushes different classes as far apart as possible. The proposed method improves downstream performance across multiple modalities, requires only few additional lines of code to implement, and introduces no computational overhead.
  arXiv  Detail & Related papers  (2020-10-09T14:18:53Z)
• Reinforced Negative Sampling over Knowledge Graph for Recommendation [106.07209348727564]
  We develop a new negative sampling model, Knowledge Graph Policy Network (kgPolicy), which works as a reinforcement learning agent to explore high-quality negatives. kgPolicy navigates from the target positive interaction, adaptively receives knowledge-aware negative signals, and ultimately yields a potential negative item to train the recommender.
  arXiv  Detail & Related papers  (2020-03-12T12:44:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.