SafeSpeech: A Comprehensive and Interactive Tool for Analysing Sexist and Abusive Language in Conversations

• URL: http://arxiv.org/abs/2503.06534v1
• Date: Sun, 09 Mar 2025 09:31:17 GMT
• Title: SafeSpeech: A Comprehensive and Interactive Tool for Analysing Sexist and Abusive Language in Conversations
• Authors: Xingwei Tan, Chen Lyu, Hafiz Muhammad Umer, Sahrish Khan, Mahathi Parvatham, Lois Arthurs, Simon Cullen, Shelley Wilson, Arshad Jhumka, Gabriele Pergola,
• Abstract summary: SafeSpeech is a comprehensive platform for toxic content detection and analysis.<n>It bridges message-level and conversation-level insights.<n>The platform integrates fine-tuned classifiers and large language models.<n> Evaluations on benchmark datasets, including EDOS, OffensEval, and HatEval, demonstrate the reproduction of state-of-the-art performance.
• Score: 7.4815142964548205
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Detecting toxic language including sexism, harassment and abusive behaviour, remains a critical challenge, particularly in its subtle and context-dependent forms. Existing approaches largely focus on isolated message-level classification, overlooking toxicity that emerges across conversational contexts. To promote and enable future research in this direction, we introduce SafeSpeech, a comprehensive platform for toxic content detection and analysis that bridges message-level and conversation-level insights. The platform integrates fine-tuned classifiers and large language models (LLMs) to enable multi-granularity detection, toxic-aware conversation summarization, and persona profiling. SafeSpeech also incorporates explainability mechanisms, such as perplexity gain analysis, to highlight the linguistic elements driving predictions. Evaluations on benchmark datasets, including EDOS, OffensEval, and HatEval, demonstrate the reproduction of state-of-the-art performance across multiple tasks, including fine-grained sexism detection.

Related papers

• PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis [74.41260927676747]
  This paper bridges the gaps by introducing a multimodal conversational Sentiment Analysis (ABSA) To benchmark the tasks, we construct PanoSent, a dataset annotated both manually and automatically, featuring high quality, large scale, multimodality, multilingualism, multi-scenarios, and covering both implicit and explicit sentiment elements. To effectively address the tasks, we devise a novel Chain-of-Sentiment reasoning framework, together with a novel multimodal large language model (namely Sentica) and a paraphrase-based verification mechanism.
  arXiv  Detail & Related papers  (2024-08-18T13:51:01Z)
• A comprehensive cross-language framework for harmful content detection with the aid of sentiment analysis [0.356008609689971]
  This study introduces, for the first time, a detailed framework adaptable to any language. A key component of the framework is the development of a general and detailed annotation guideline. The integration of sentiment analysis represents a novel approach to enhancing harmful language detection.
  arXiv  Detail & Related papers  (2024-03-02T17:13:47Z)
• Paralinguistics-Enhanced Large Language Modeling of Spoken Dialogue [71.15186328127409]
  Paralinguistics-enhanced Generative Pretrained Transformer (ParalinGPT) Model takes the conversational context of text, speech embeddings, and paralinguistic attributes as input prompts within a serialized multitasking framework. We utilize the Switchboard-1 corpus, including its sentiment labels as the paralinguistic attribute, as our spoken dialogue dataset.
  arXiv  Detail & Related papers  (2023-12-23T18:14:56Z)
• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• Exploiting Explainability to Design Adversarial Attacks and Evaluate Attack Resilience in Hate-Speech Detection Models [0.47334880432883714]
  We present an analysis of adversarial robustness exhibited by various hate-speech detection models. We devise and execute targeted attacks on the text by leveraging the TextAttack tool. This work paves the way for creating more robust and reliable hate-speech detection systems.
  arXiv  Detail & Related papers  (2023-05-29T19:59:40Z)
• Contextual Object Detection with Multimodal Large Language Models [66.15566719178327]
  We introduce a novel research problem of contextual object detection. Three representative scenarios are investigated, including the language cloze test, visual captioning, and question answering. We present ContextDET, a unified multimodal model that is capable of end-to-end differentiable modeling of visual-language contexts.
  arXiv  Detail & Related papers  (2023-05-29T17:50:33Z)
• Countering Malicious Content Moderation Evasion in Online Social Networks: Simulation and Detection of Word Camouflage [64.78260098263489]
  Twisting and camouflaging keywords are among the most used techniques to evade platform content moderation systems. This article contributes significantly to countering malicious information by developing multilingual tools to simulate and detect new methods of evasion of content.
  arXiv  Detail & Related papers  (2022-12-27T16:08:49Z)
• Speech Toxicity Analysis: A New Spoken Language Processing Task [32.297717021285344]
  Toxic speech, also known as hate speech, is regarded as one of the crucial issues plaguing online social media today. We propose a new Spoken Language Processing task of detecting toxicity from spoken speech. We introduce DeToxy, the first publicly available toxicity annotated dataset for English speech, sourced from various openly available speech databases.
  arXiv  Detail & Related papers  (2021-10-14T17:51:04Z)
• Topic-Aware Multi-turn Dialogue Modeling [91.52820664879432]
  This paper presents a novel solution for multi-turn dialogue modeling, which segments and extracts topic-aware utterances in an unsupervised way. Our topic-aware modeling is implemented by a newly proposed unsupervised topic-aware segmentation algorithm and Topic-Aware Dual-attention Matching (TADAM) Network.
  arXiv  Detail & Related papers  (2020-09-26T08:43:06Z)
• "To Target or Not to Target": Identification and Analysis of Abusive Text Using Ensemble of Classifiers [18.053219155702465]
  We present an ensemble learning method to identify and analyze abusive and hateful content on social media platforms. Our stacked ensemble comprises of three machine learning models that capture different aspects of language and provide diverse and coherent insights about inappropriate language.
  arXiv  Detail & Related papers  (2020-06-05T06:59:22Z)
• RECAST: Interactive Auditing of Automatic Toxicity Detection Models [39.621867230707814]
  We present our ongoing work, RECAST, an interactive tool for examining toxicity detection models by visualizing explanations for predictions and providing alternative wordings for detected toxic speech.
  arXiv  Detail & Related papers  (2020-01-07T00:17:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.