HABERTOR: An Efficient and Effective Deep Hatespeech Detector

• URL: http://arxiv.org/abs/2010.08865v1
• Date: Sat, 17 Oct 2020 21:10:08 GMT
• Title: HABERTOR: An Efficient and Effective Deep Hatespeech Detector
• Authors: Thanh Tran, Yifan Hu, Changwei Hu, Kevin Yen, Fei Tan, Kyumin Lee, Serim Park
• Abstract summary: We present our HABERTOR model for detecting hatespeech in user-generated content. We show that HABERTOR works better than 15 state-of-the-art hatespeech detection methods. Our generalizability analysis shows that HABERTOR transfers well to other unseen hatespeech datasets.
• Score: 14.315255338162283
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present our HABERTOR model for detecting hatespeech in large scale user-generated content. Inspired by the recent success of the BERT model, we propose several modifications to BERT to enhance the performance on the downstream hatespeech classification task. HABERTOR inherits BERT's architecture, but is different in four aspects: (i) it generates its own vocabularies and is pre-trained from the scratch using the largest scale hatespeech dataset; (ii) it consists of Quaternion-based factorized components, resulting in a much smaller number of parameters, faster training and inferencing, as well as less memory usage; (iii) it uses our proposed multi-source ensemble heads with a pooling layer for separate input sources, to further enhance its effectiveness; and (iv) it uses a regularized adversarial training with our proposed fine-grained and adaptive noise magnitude to enhance its robustness. Through experiments on the large-scale real-world hatespeech dataset with 1.4M annotated comments, we show that HABERTOR works better than 15 state-of-the-art hatespeech detection methods, including fine-tuning Language Models. In particular, comparing with BERT, our HABERTOR is 4~5 times faster in the training/inferencing phase, uses less than 1/3 of the memory, and has better performance, even though we pre-train it by using less than 1% of the number of words. Our generalizability analysis shows that HABERTOR transfers well to other unseen hatespeech datasets and is a more efficient and effective alternative to BERT for the hatespeech classification.

Related papers

• Counterspeech the ultimate shield! Multi-Conditioned Counterspeech Generation through Attributed Prefix Learning [20.199270923708042]
  HiPPrO, Hierarchical Prefix learning with Preference Optimization, is a novel framework for generating constructive counterspeech.<n>We show that HiPPrO achieves a 38 % improvement in intent conformity and a 3 %, 2 %, 3 %, 3 % improvement in Rouge-1, Rouge-2, and Rouge-L, respectively.
  arXiv  Detail & Related papers  (2025-05-17T11:19:49Z)
• Cross-Lingual Query-by-Example Spoken Term Detection: A Transformer-Based Approach [0.0]
  This paper introduces a novel, language-agnostic QbE-STD model leveraging image processing techniques and transformer architecture. Experimental results across four languages demonstrate significant performance gains (19-54%) over a CNN-based baseline.
  arXiv  Detail & Related papers  (2024-10-05T09:19:29Z)
• BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models [77.0501668780182]
  Retrieval augmentation addresses many critical problems in large language models. Running retrieval-augmented language models (LMs) is slow and difficult to scale due to processing large amounts of retrieved text. We introduce binary token representations (BTR), which use 1-bit vectors to precompute every token in passages.
  arXiv  Detail & Related papers  (2023-10-02T16:48:47Z)
• LongFNT: Long-form Speech Recognition with Factorized Neural Transducer [64.75547712366784]
  We propose the LongFNT-Text architecture, which fuses the sentence-level long-form features directly with the output of the vocabulary predictor. The effectiveness of our LongFNT approach is validated on LibriSpeech and GigaSpeech corpora with 19% and 12% relative word error rate(WER) reduction, respectively.
  arXiv  Detail & Related papers  (2022-11-17T08:48:27Z)
• Simple and Effective Unsupervised Speech Translation [68.25022245914363]
  We study a simple and effective approach to build speech translation systems without labeled data. We present an unsupervised domain adaptation technique for pre-trained speech models. Experiments show that unsupervised speech-to-text translation outperforms the previous unsupervised state of the art.
  arXiv  Detail & Related papers  (2022-10-18T22:26:13Z)
• Robustification of Multilingual Language Models to Real-world Noise with Robust Contrastive Pretraining [14.087882550564169]
  We assess the robustness of neural models on noisy data and suggest improvements are limited to the English language. To benchmark the performance of pretrained multilingual models, we construct noisy datasets covering five languages and four NLP tasks. We propose Robust Contrastive Pretraining (RCP) to boost the zero-shot cross-lingual robustness of multilingual pretrained models.
  arXiv  Detail & Related papers  (2022-10-10T15:40:43Z)
• Speaker Embedding-aware Neural Diarization: a Novel Framework for Overlapped Speech Diarization in the Meeting Scenario [51.5031673695118]
  We reformulate overlapped speech diarization as a single-label prediction problem. We propose the speaker embedding-aware neural diarization (SEND) system.
  arXiv  Detail & Related papers  (2022-03-18T06:40:39Z)
• BERT-LID: Leveraging BERT to Improve Spoken Language Identification [12.179375898668614]
  Language identification is a task of automatically determining the identity of a language conveyed by a spoken segment. Despite language identification attaining high accuracy on medium or long utterances, the performance on short utterances is still far from satisfactory. We propose an effective BERT-based language identification system (BERT-LID) to improve language identification performance.
  arXiv  Detail & Related papers  (2022-03-01T10:01:25Z)
• Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction [109.44933866397123]
  Noise robustness is essential for deploying automatic speech recognition systems in real-world environments. We employ a noise-robust representation learned by a refined self-supervised framework for noisy speech recognition. We achieve comparable performance to the best supervised approach reported with only 16% of labeled data.
  arXiv  Detail & Related papers  (2021-10-28T20:39:02Z)
• Wav2vec-Switch: Contrastive Learning from Original-noisy Speech Pairs for Robust Speech Recognition [52.71604809100364]
  We propose wav2vec-Switch, a method to encode noise robustness into contextualized representations of speech. Specifically, we feed original-noisy speech pairs simultaneously into the wav2vec 2.0 network. In addition to the existing contrastive learning task, we switch the quantized representations of the original and noisy speech as additional prediction targets.
  arXiv  Detail & Related papers  (2021-10-11T00:08:48Z)
• To BAN or not to BAN: Bayesian Attention Networks for Reliable Hate Speech Detection [3.7768834126209234]
  Hate speech is an important problem in the management of user-generated content. To remove offensive content or ban misbehaving users, content moderators need reliable hate speech detectors. Deep neural networks based on the transformer architecture, such as the (multilingual) BERT model, achieve superior performance in many natural language classification tasks, including hate speech detection. We propose a Bayesian method using Monte Carlo dropout within the attention layers of the transformer models to provide well-calibrated reliability estimates.
  arXiv  Detail & Related papers  (2020-07-10T11:09:00Z)
• An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features [13.97006782398121]
  Bidirectional Representations from Transformers (BERT) model was proposed and has achieved record-breaking success on many natural language processing tasks. We explore the incorporation of confidence scores into sentence representations to see if such an attempt could help alleviate the negative effects caused by imperfect automatic speech recognition. We validate the effectiveness of our proposed method on a benchmark dataset.
  arXiv  Detail & Related papers  (2020-06-01T18:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.