Overview of Abusive and Threatening Language Detection in Urdu at FIRE 2021

• URL: http://arxiv.org/abs/2207.06710v1
• Date: Thu, 14 Jul 2022 07:38:13 GMT
• Title: Overview of Abusive and Threatening Language Detection in Urdu at FIRE 2021
• Authors: Maaz Amjad, Alisa Zhila, Grigori Sidorov, Andrey Labunets, Sabur Butta, Hamza Imam Amjad, Oxana Vitman, Alexander Gelbukh
• Abstract summary: We present two shared tasks of abusive and threatening language detection for the Urdu language. We present two manually annotated datasets containing tweets labelled as (i) Abusive and Non-Abusive, and (ii) Threatening and Non-Threatening. For both subtasks, m-Bert based transformer model showed the best performance.
• Score: 50.591267188664666
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the growth of social media platform influence, the effect of their misuse becomes more and more impactful. The importance of automatic detection of threatening and abusive language can not be overestimated. However, most of the existing studies and state-of-the-art methods focus on English as the target language, with limited work on low- and medium-resource languages. In this paper, we present two shared tasks of abusive and threatening language detection for the Urdu language which has more than 170 million speakers worldwide. Both are posed as binary classification tasks where participating systems are required to classify tweets in Urdu into two classes, namely: (i) Abusive and Non-Abusive for the first task, and (ii) Threatening and Non-Threatening for the second. We present two manually annotated datasets containing tweets labelled as (i) Abusive and Non-Abusive, and (ii) Threatening and Non-Threatening. The abusive dataset contains 2400 annotated tweets in the train part and 1100 annotated tweets in the test part. The threatening dataset contains 6000 annotated tweets in the train part and 3950 annotated tweets in the test part. We also provide logistic regression and BERT-based baseline classifiers for both tasks. In this shared task, 21 teams from six countries registered for participation (India, Pakistan, China, Malaysia, United Arab Emirates, and Taiwan), 10 teams submitted their runs for Subtask A, which is Abusive Language Detection and 9 teams submitted their runs for Subtask B, which is Threatening Language detection, and seven teams submitted their technical reports. The best performing system achieved an F1-score value of 0.880 for Subtask A and 0.545 for Subtask B. For both subtasks, m-Bert based transformer model showed the best performance.

Related papers

• Overview of the 2023 ICON Shared Task on Gendered Abuse Detection in Indic Languages [7.869644160487393]
  The paper reports the findings of the ICON 2023 on Gendered Abuse Detection in Indic languages. The shared task was conducted based on a novel dataset in Hindi, Tamil and the Indian dialect of English. The paper contains examples of hateful content owing to its topic.
  arXiv  Detail & Related papers  (2024-01-08T05:54:26Z)
• Overview of the Shared Task on Fake News Detection in Urdu at FIRE 2020 [62.6928395368204]
  Task was posed as a binary classification task, in which the goal is to differentiate between real and fake news. We provided a dataset divided into 900 annotated news articles for training and 400 news articles for testing. 42 teams from 6 different countries (India, China, Egypt, Germany, Pakistan, and the UK) registered for the task.
  arXiv  Detail & Related papers  (2022-07-25T03:41:32Z)
• UrduFake@FIRE2021: Shared Track on Fake News Identification in Urdu [55.41644538483948]
  This study reports the second shared task named as UrduFake@FIRE2021 on identifying fake news detection in Urdu language. The proposed systems were based on various count-based features and used different classifiers as well as neural network architectures. The gradient descent (SGD) algorithm outperformed other classifiers and achieved 0.679 F-score.
  arXiv  Detail & Related papers  (2022-07-11T19:15:04Z)
• Overview of the Shared Task on Fake News Detection in Urdu at FIRE 2021 [55.41644538483948]
  The goal of the shared task is to motivate the community to come up with efficient methods for solving this vital problem. The training set contains 1300 annotated news articles -- 750 real news, 550 fake news, while the testing set contains 300 news articles -- 200 real, 100 fake news. The best performing system obtained an F1-macro score of 0.679, which is lower than the past year's best result of 0.907 F1-macro.
  arXiv  Detail & Related papers  (2022-07-11T18:58:36Z)
• No Language Left Behind: Scaling Human-Centered Machine Translation [69.28110770760506]
  We create datasets and models aimed at narrowing the performance gap between low and high-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-11T07:33:36Z)
• Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages [4.267837363677351]
  This paper presents the HASOC subtrack for English, Hindi, and Marathi. The data set was assembled from Twitter. The performance of the best classification algorithms for task A are F1 measures 0.91, 0.78 and 0.83 for Marathi, Hindi and English, respectively.
  arXiv  Detail & Related papers  (2021-12-17T03:28:54Z)
• Abusive and Threatening Language Detection in Urdu using Boosting based and BERT based models: A Comparative Approach [0.0]
  In this paper, we explore several machine learning models for abusive and threatening content detection in Urdu based on the shared task. Our model came First for both abusive and threatening content detection with an F1scoreof 0.88 and 0.54, respectively.
  arXiv  Detail & Related papers  (2021-11-27T20:03:19Z)
• Fine-tuning of Pre-trained Transformers for Hate, Offensive, and Profane Content Detection in English and Marathi [0.0]
  This paper describes neural models developed for the Hate Speech and Offensive Content Identification in English and Indo-Aryan languages. For English subtasks, we investigate the impact of additional corpora for hate speech detection to fine-tune transformer models. For the Marathi tasks, we propose a system based on the Language-Agnostic BERT Sentence Embedding (LaBSE)
  arXiv  Detail & Related papers  (2021-10-25T07:11:02Z)
• Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection [55.445023584632175]
  We build an offensive language detection system, which combines multi-task learning with BERT-based models. Our model achieves 91.51% F1 score in English Sub-task A, which is comparable to the first place.
  arXiv  Detail & Related papers  (2020-04-28T11:27:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.