cantnlp@LT-EDI-2023: Homophobia/Transphobia Detection in Social Media Comments using Spatio-Temporally Retrained Language Models

• URL: http://arxiv.org/abs/2308.10370v2
• Date: Fri, 25 Aug 2023 01:41:17 GMT
• Title: cantnlp@LT-EDI-2023: Homophobia/Transphobia Detection in Social Media Comments using Spatio-Temporally Retrained Language Models
• Authors: Sidney G.-J. Wong, Matthew Durward, Benjamin Adams and Jonathan Dunn
• Abstract summary: This paper describes our multiclass classification system developed as part of the LTERAN@LP-2023 shared task. We used a BERT-based language model to detect homophobic and transphobic content in social media comments across five language conditions. We developed the best performing seven-label classification system for Malayalam based on weighted macro averaged F1 score.
• Score: 0.9012198585960441
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper describes our multiclass classification system developed as part of the LTEDI@RANLP-2023 shared task. We used a BERT-based language model to detect homophobic and transphobic content in social media comments across five language conditions: English, Spanish, Hindi, Malayalam, and Tamil. We retrained a transformer-based crosslanguage pretrained language model, XLMRoBERTa, with spatially and temporally relevant social media language data. We also retrained a subset of models with simulated script-mixed social media language data with varied performance. We developed the best performing seven-label classification system for Malayalam based on weighted macro averaged F1 score (ranked first out of six) with variable performance for other language and class-label conditions. We found the inclusion of this spatio-temporal data improved the classification performance for all language and task conditions when compared with the baseline. The results suggests that transformer-based language classification systems are sensitive to register-specific and language-specific retraining.

Related papers

• cantnlp@LT-EDI-2024: Automatic Detection of Anti-LGBTQ+ Hate Speech in Under-resourced Languages [0.0]
  This paper describes our homophobia/transphobia in social media comments detection system developed as part of the shared task at LT-EDI-2024. We took a transformer-based approach to develop our multiclass classification model for ten language conditions. We introduced synthetic and organic instances of script-switched language data during domain adaptation to mirror the linguistic realities of social media language.
  arXiv  Detail & Related papers  (2024-01-28T21:58:04Z)
• T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classification [50.675552118811]
  Cross-lingual text classification is typically built on large-scale, multilingual language models (LMs) pretrained on a variety of languages of interest. We propose revisiting the classic "translate-and-test" pipeline to neatly separate the translation and classification stages.
  arXiv  Detail & Related papers  (2023-06-08T07:33:22Z)
• Leveraging Language Identification to Enhance Code-Mixed Text Classification [0.7340017786387767]
  Existing deep-learning models do not take advantage of the implicit language information in code-mixed text. Our study aims to improve BERT-based models performance on low-resource Code-Mixed Hindi-English datasets.
  arXiv  Detail & Related papers  (2023-06-08T06:43:10Z)
• Comparative Study of Pre-Trained BERT Models for Code-Mixed Hindi-English Data [0.7874708385247353]
  "Code Mixed" refers to the use of more than one language in the same text. In this work, we focus on low-resource Hindi-English code-mixed language. We report state-of-the-art results on respective datasets using HingBERT-based models.
  arXiv  Detail & Related papers  (2023-05-25T05:10:28Z)
• Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval [109.62363167257664]
  We propose a generative model for learning multilingual text embeddings. Our model operates on parallel data in $N$ languages. We evaluate this method on a suite of tasks including semantic similarity, bitext mining, and cross-lingual question retrieval.
  arXiv  Detail & Related papers  (2022-12-21T02:41:40Z)
• Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval [66.69799641522133]
  State-of-the-art neural (re)rankers are notoriously data hungry. Current approaches typically transfer rankers trained on English data to other languages and cross-lingual setups by means of multilingual encoders. We show that two parameter-efficient approaches to cross-lingual transfer, namely Sparse Fine-Tuning Masks (SFTMs) and Adapters, allow for a more lightweight and more effective zero-shot transfer.
  arXiv  Detail & Related papers  (2022-04-05T15:44:27Z)
• Mixed Attention Transformer for LeveragingWord-Level Knowledge to Neural Cross-Lingual Information Retrieval [15.902630454568811]
  We propose a novel Mixed Attention Transformer (MAT) that incorporates external word level knowledge, such as a dictionary or translation table. By encoding the translation knowledge into an attention matrix, the model with MAT is able to focus on the mutually translated words in the input sequence.
  arXiv  Detail & Related papers  (2021-09-07T00:33:14Z)
• WangchanBERTa: Pretraining transformer-based Thai Language Models [2.186960190193067]
  We pretrain a language model based on RoBERTa-base architecture on a large, deduplicated, cleaned training set (78GB in total size) We apply text processing rules that are specific to Thai most importantly preserving spaces. We also experiment with word-level, syllable-level and SentencePiece tokenization with a smaller dataset to explore the effects on tokenization on downstream performance.
  arXiv  Detail & Related papers  (2021-01-24T03:06:34Z)
• VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation [77.82373082024934]
  We plug a cross-attention module into the Transformer encoder to explicitly build the interdependence between languages. It can effectively avoid the degeneration of predicting masked words only conditioned on the context in its own language. The proposed cross-lingual model delivers new state-of-the-art results on various cross-lingual understanding tasks of the XTREME benchmark.
  arXiv  Detail & Related papers  (2020-10-30T03:41:38Z)
• Comparison of Interactive Knowledge Base Spelling Correction Models for Low-Resource Languages [81.90356787324481]
  Spelling normalization for low resource languages is a challenging task because the patterns are hard to predict. This work shows a comparison of a neural model and character language models with varying amounts on target language data. Our usage scenario is interactive correction with nearly zero amounts of training examples, improving models as more data is collected.
  arXiv  Detail & Related papers  (2020-10-20T17:31:07Z)
• Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification [52.69730591919885]
  We present a semi-supervised adversarial training process that minimizes the maximal loss for label-preserving input perturbations. We observe significant gains in effectiveness on document and intent classification for a diverse set of languages.
  arXiv  Detail & Related papers  (2020-07-29T19:38:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.