Task Adaptive Pretraining of Transformers for Hostility Detection

• URL: http://arxiv.org/abs/2101.03382v1
• Date: Sat, 9 Jan 2021 15:45:26 GMT
• Title: Task Adaptive Pretraining of Transformers for Hostility Detection
• Authors: Tathagata Raha, Sayar Ghosh Roy, Ujwal Narayan, Zubair Abid, Vasudeva Varma
• Abstract summary: We study two problems, namely, (a) Coarse binary classification of Hindi Tweets into Hostile or Not, and (b) Fine-grained multi-label classification of Tweets into four categories: hate, fake, offensive, and defamation. Our system ranked first in the 'Hostile Post Detection in Hindi' shared task with an F1 score of 97.16% for coarse-grained detection and a weighted F1 score of 62.96% for fine-grained multi-label classification on the provided blind test corpora.
• Score: 11.306581296760864
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identifying adverse and hostile content on the web and more particularly, on social media, has become a problem of paramount interest in recent years. With their ever increasing popularity, fine-tuning of pretrained Transformer-based encoder models with a classifier head are gradually becoming the new baseline for natural language classification tasks. In our work, we explore the gains attributed to Task Adaptive Pretraining (TAPT) prior to fine-tuning of Transformer-based architectures. We specifically study two problems, namely, (a) Coarse binary classification of Hindi Tweets into Hostile or Not, and (b) Fine-grained multi-label classification of Tweets into four categories: hate, fake, offensive, and defamation. Building up on an architecture which takes emojis and segmented hashtags into consideration for classification, we are able to experimentally showcase the performance upgrades due to TAPT. Our system (with team name 'iREL IIIT') ranked first in the 'Hostile Post Detection in Hindi' shared task with an F1 score of 97.16% for coarse-grained detection and a weighted F1 score of 62.96% for fine-grained multi-label classification on the provided blind test corpora.

Related papers

• STF: Sentence Transformer Fine-Tuning For Topic Categorization With Limited Data [0.27309692684728604]
  Sentence Transformers Fine-tuning (STF) is a topic detection system that leverages pretrained Sentence Transformers models and fine-tuning to classify topics from tweets accurately. Our main contribution is the achievement of promising results in tweet topic classification by applying pretrained sentence transformers language models.
  arXiv  Detail & Related papers  (2024-07-03T16:34:56Z)
• BERT Goes Off-Topic: Investigating the Domain Transfer Challenge using Genre Classification [0.27195102129095]
  We show that classification tasks still suffer from a performance gap when the underlying distribution of topics changes. We quantify this phenomenon empirically with a large corpus and a large set of topics. We suggest and successfully test a possible remedy: after augmenting the training dataset with topically-controlled synthetic texts, the F1 score improves by up to 50% for some topics.
  arXiv  Detail & Related papers  (2023-11-27T18:53:31Z)
• Balanced Classification: A Unified Framework for Long-Tailed Object Detection [74.94216414011326]
  Conventional detectors suffer from performance degradation when dealing with long-tailed data due to a classification bias towards the majority head categories. We introduce a unified framework called BAlanced CLassification (BACL), which enables adaptive rectification of inequalities caused by disparities in category distribution. BACL consistently achieves performance improvements across various datasets with different backbones and architectures.
  arXiv  Detail & Related papers  (2023-08-04T09:11:07Z)
• Like a Good Nearest Neighbor: Practical Content Moderation and Text Classification [66.02091763340094]
  Like a Good Nearest Neighbor (LaGoNN) is a modification to SetFit that introduces no learnable parameters but alters input text with information from its nearest neighbor. LaGoNN is effective at flagging undesirable content and text classification, and improves the performance of SetFit.
  arXiv  Detail & Related papers  (2023-02-17T15:43:29Z)
• Subsidiary Prototype Alignment for Universal Domain Adaptation [58.431124236254]
  A major problem in Universal Domain Adaptation (UniDA) is misalignment of "known" and "unknown" classes. We propose a novel word-histogram-related pretext task to enable closed-set SPA, operating in conjunction with goal task UniDA. We demonstrate the efficacy of our approach on top of existing UniDA techniques, yielding state-of-the-art performance across three standard UniDA and Open-Set DA object recognition benchmarks.
  arXiv  Detail & Related papers  (2022-10-28T05:32:14Z)
• Paragraph-based Transformer Pre-training for Multi-Sentence Inference [99.59693674455582]
  We show that popular pre-trained transformers perform poorly when used for fine-tuning on multi-candidate inference tasks. We then propose a new pre-training objective that models the paragraph-level semantics across multiple input sentences.
  arXiv  Detail & Related papers  (2022-05-02T21:41:14Z)
• Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classification [59.698811329287174]
  We leverage GPT-2 for generating artificial training instances in order to improve classification performance. Our results show that fine-tuning GPT-2 in a handful of label instances leads to consistent classification improvements.
  arXiv  Detail & Related papers  (2021-11-17T12:10:03Z)
• Walk in Wild: An Ensemble Approach for Hostility Detection in Hindi Posts [3.9373541926236766]
  We develop a simple ensemble based model on pre-trained mBERT and popular classification algorithms like Artificial Neural Network (ANN) and XGBoost for hostility detection in Hindi posts. We received third overall rank in the competition and weighted F1-scores of 0.969 and 0.61 on the binary and multi-label multi-class classification tasks respectively.
  arXiv  Detail & Related papers  (2021-01-15T07:49:27Z)
• Bangla Text Classification using Transformers [2.3475904942266697]
  Text classification has been one of the earliest problems in NLP. In this work, we fine-tune multilingual Transformer models for Bangla text classification tasks. We obtain the state of the art results on six benchmark datasets, improving upon the previous results by 5-29% accuracy across different tasks.
  arXiv  Detail & Related papers  (2020-11-09T14:12:07Z)
• Rank over Class: The Untapped Potential of Ranking in Natural Language Processing [8.637110868126546]
  We argue that many tasks which are currently addressed using classification are in fact being shoehorned into a classification mould. We propose a novel end-to-end ranking approach consisting of a Transformer network responsible for producing representations for a pair of text sequences. In an experiment on a heavily-skewed sentiment analysis dataset, converting ranking results to classification labels yields an approximately 22% improvement over state-of-the-art text classification.
  arXiv  Detail & Related papers  (2020-09-10T22:18:57Z)
• Fine-Grained Visual Classification with Efficient End-to-end Localization [49.9887676289364]
  We present an efficient localization module that can be fused with a classification network in an end-to-end setup. We evaluate the new model on the three benchmark datasets CUB200-2011, Stanford Cars and FGVC-Aircraft.
  arXiv  Detail & Related papers  (2020-05-11T14:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.