Leveraging Affective Bidirectional Transformers for Offensive Language
Detection
- URL: http://arxiv.org/abs/2006.01266v1
- Date: Sat, 16 May 2020 04:55:35 GMT
- Title: Leveraging Affective Bidirectional Transformers for Offensive Language
Detection
- Authors: AbdelRahim Elmadany, Chiyu Zhang, Muhammad Abdul-Mageed, Azadeh
Hashemi
- Abstract summary: We focus on developing purely deep learning systems, without a need for feature engineering.
Our best models are significantly better than a vanilla BERT model, with 89.60% acc (82.31% macro F1) for hate speech and 95.20% acc (70.51% macro F1) on official TEST data.
- Score: 7.09232719022402
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Social media are pervasive in our life, making it necessary to ensure safe
online experiences by detecting and removing offensive and hate speech. In this
work, we report our submission to the Offensive Language and hate-speech
Detection shared task organized with the 4th Workshop on Open-Source Arabic
Corpora and Processing Tools Arabic (OSACT4). We focus on developing purely
deep learning systems, without a need for feature engineering. For that
purpose, we develop an effective method for automatic data augmentation and
show the utility of training both offensive and hate speech models off (i.e.,
by fine-tuning) previously trained affective models (i.e., sentiment and
emotion). Our best models are significantly better than a vanilla BERT model,
with 89.60% acc (82.31% macro F1) for hate speech and 95.20% acc (70.51% macro
F1) on official TEST data.
Related papers
- Bag of Tricks for Effective Language Model Pretraining and Downstream
Adaptation: A Case Study on GLUE [93.98660272309974]
This report briefly describes our submission Vega v1 on the General Language Understanding Evaluation leaderboard.
GLUE is a collection of nine natural language understanding tasks, including question answering, linguistic acceptability, sentiment analysis, text similarity, paraphrase detection, and natural language inference.
With our optimized pretraining and fine-tuning strategies, our 1.3 billion model sets new state-of-the-art on 4/9 tasks, achieving the best average score of 91.3.
arXiv Detail & Related papers (2023-02-18T09:26:35Z) - Hate Speech and Offensive Language Detection using an Emotion-aware
Shared Encoder [1.8734449181723825]
Existing works on hate speech and offensive language detection produce promising results based on pre-trained transformer models.
This paper addresses a multi-task joint learning approach which combines external emotional features extracted from another corpora.
Our findings demonstrate that emotional knowledge helps to more reliably identify hate speech and offensive language across datasets.
arXiv Detail & Related papers (2023-02-17T09:31:06Z) - From English to More Languages: Parameter-Efficient Model Reprogramming
for Cross-Lingual Speech Recognition [50.93943755401025]
We propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition.
We design different auxiliary neural architectures focusing on learnable pre-trained feature enhancement.
Our methods outperform existing ASR tuning architectures and their extension with self-supervised losses.
arXiv Detail & Related papers (2023-01-19T02:37:56Z) - Toward Efficient Language Model Pretraining and Downstream Adaptation
via Self-Evolution: A Case Study on SuperGLUE [203.65227947509933]
This report describes our JDExplore d-team's Vega v2 submission on the SuperGLUE leaderboard.
SuperGLUE is more challenging than the widely used general language understanding evaluation (GLUE) benchmark, containing eight difficult language understanding tasks.
arXiv Detail & Related papers (2022-12-04T15:36:18Z) - Multilingual Hate Speech and Offensive Content Detection using Modified
Cross-entropy Loss [0.0]
Large language models are trained on a lot of data and they also make use of contextual embeddings.
The data is also quite unbalanced; so we used a modified cross-entropy loss to tackle the issue.
Our team (HNLP) achieved the macro F1-scores of 0.808, 0.639 in English Subtask A and English Subtask B respectively.
arXiv Detail & Related papers (2022-02-05T20:31:40Z) - LaMDA: Language Models for Dialog Applications [75.75051929981933]
LaMDA is a family of Transformer-based neural language models specialized for dialog.
Fine-tuning with annotated data and enabling the model to consult external knowledge sources can lead to significant improvements.
arXiv Detail & Related papers (2022-01-20T15:44:37Z) - Addressing the Challenges of Cross-Lingual Hate Speech Detection [115.1352779982269]
In this paper we focus on cross-lingual transfer learning to support hate speech detection in low-resource languages.
We leverage cross-lingual word embeddings to train our neural network systems on the source language and apply it to the target language.
We investigate the issue of label imbalance of hate speech datasets, since the high ratio of non-hate examples compared to hate examples often leads to low model performance.
arXiv Detail & Related papers (2022-01-15T20:48:14Z) - Offensive Language and Hate Speech Detection with Deep Learning and
Transfer Learning [1.77356577919977]
We propose an approach to automatically classify tweets into three classes: Hate, offensive and Neither.
We create a class module which contains main functionality including text classification, sentiment checking and text data augmentation.
arXiv Detail & Related papers (2021-08-06T20:59:47Z) - HASOCOne@FIRE-HASOC2020: Using BERT and Multilingual BERT models for
Hate Speech Detection [9.23545668304066]
We propose an approach to automatically classify hate speech and offensive content.
We have used the datasets obtained from FIRE 2019 and 2020 shared tasks.
We observed that the pre-trained BERT model and the multilingual-BERT model gave the best results.
arXiv Detail & Related papers (2021-01-22T08:55:32Z) - Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for
Offensive Language Detection [55.445023584632175]
We build an offensive language detection system, which combines multi-task learning with BERT-based models.
Our model achieves 91.51% F1 score in English Sub-task A, which is comparable to the first place.
arXiv Detail & Related papers (2020-04-28T11:27:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.