Revisiting Contextual Toxicity Detection in Conversations

• URL: http://arxiv.org/abs/2111.12447v1
• Date: Wed, 24 Nov 2021 11:50:37 GMT
• Title: Revisiting Contextual Toxicity Detection in Conversations
• Authors: Julia Ive, Atijit Anuchitanukul and Lucia Specia
• Abstract summary: We show that toxicity labelling by humans is in general influenced by the conversational structure, polarity and topic of the context. We propose to bring these findings into computational detection models by introducing (a) neural architectures for contextual toxicity detection. We have also demonstrated that such models can benefit from synthetic data, especially in the social media domain.
• Score: 28.465019968374413
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding toxicity in user conversations is undoubtedly an important problem. As it has been argued in previous work, addressing "covert" or implicit cases of toxicity is particularly hard and requires context. Very few previous studies have analysed the influence of conversational context in human perception or in automated detection models. We dive deeper into both these directions. We start by analysing existing contextual datasets and come to the conclusion that toxicity labelling by humans is in general influenced by the conversational structure, polarity and topic of the context. We then propose to bring these findings into computational detection models by introducing (a) neural architectures for contextual toxicity detection that are aware of the conversational structure, and (b) data augmentation strategies that can help model contextual toxicity detection. Our results have shown the encouraging potential of neural architectures that are aware of the conversation structure. We have also demonstrated that such models can benefit from synthetic data, especially in the social media domain.

Related papers

• How Well Do Text Embedding Models Understand Syntax? [50.440590035493074]
  The ability of text embedding models to generalize across a wide range of syntactic contexts remains under-explored. Our findings reveal that existing text embedding models have not sufficiently addressed these syntactic understanding challenges. We propose strategies to augment the generalization ability of text embedding models in diverse syntactic scenarios.
  arXiv  Detail & Related papers  (2023-11-14T08:51:00Z)
• ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation [43.356758428820626]
  We introduce ToxicChat, a novel benchmark based on real user queries from an open-source chatbots. Our systematic evaluation of models trained on existing toxicity datasets has shown their shortcomings when applied to this unique domain of ToxicChat. In the future, ToxicChat can be a valuable resource to drive further advancements toward building a safe and healthy environment for user-AI interactions.
  arXiv  Detail & Related papers  (2023-10-26T13:35:41Z)
• CausalDialogue: Modeling Utterance-level Causality in Conversations [83.03604651485327]
  We have compiled and expanded upon a new dataset called CausalDialogue through crowd-sourcing. This dataset includes multiple cause-effect pairs within a directed acyclic graph (DAG) structure. We propose a causality-enhanced method called Exponential Average Treatment Effect (ExMATE) to enhance the impact of causality at the utterance level in training neural conversation models.
  arXiv  Detail & Related papers  (2022-12-20T18:31:50Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• Toxicity Detection can be Sensitive to the Conversational Context [64.28043776806213]
  We construct and publicly release a dataset of 10,000 posts with two kinds of toxicity labels. We introduce a new task, context sensitivity estimation, which aims to identify posts whose perceived toxicity changes if the context is also considered.
  arXiv  Detail & Related papers  (2021-11-19T13:57:26Z)
• Mitigating Biases in Toxic Language Detection through Invariant Rationalization [70.36701068616367]
  biases toward some attributes, including gender, race, and dialect, exist in most training datasets for toxicity detection. We propose to use invariant rationalization (InvRat), a game-theoretic framework consisting of a rationale generator and a predictor, to rule out the spurious correlation of certain syntactic patterns. Our method yields lower false positive rate in both lexical and dialectal attributes than previous debiasing methods.
  arXiv  Detail & Related papers  (2021-06-14T08:49:52Z)
• CONDA: a CONtextual Dual-Annotated dataset for in-game toxicity understanding and detection [1.6085428542036968]
  CONDA is a new dataset for in-game toxic language detection enabling joint intent classification and slot filling analysis. The dataset consists of 45K utterances from 12K conversations from the chat logs of 1.9K completed Dota 2 matches. A thorough in-game toxicity analysis provides comprehensive understanding of context at utterance, token, and dual levels.
  arXiv  Detail & Related papers  (2021-06-11T07:42:12Z)
• I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling [104.09033240889106]
  We introduce the DialoguE COntradiction DEtection task (DECODE) and a new conversational dataset containing both human-human and human-bot contradictory dialogues. We then compare a structured utterance-based approach of using pre-trained Transformer models for contradiction detection with the typical unstructured approach.
  arXiv  Detail & Related papers  (2020-12-24T18:47:49Z)
• Using Sentiment Information for Preemptive Detection of Toxic Comments in Online Conversations [0.0]
  Some authors have tried to predict if a conversation will derail into toxicity using the features of the first few messages. We show how the sentiments expressed in the first messages of a conversation can help predict upcoming toxicity.
  arXiv  Detail & Related papers  (2020-06-17T20:41:57Z)
• BiERU: Bidirectional Emotional Recurrent Unit for Conversational Sentiment Analysis [18.1320976106637]
  The main difference between conversational sentiment analysis and single sentence sentiment analysis is the existence of context information. Existing approaches employ complicated deep learning structures to distinguish different parties in a conversation and then model the context information. We propose a fast, compact and parameter-efficient party-ignorant framework named bidirectional emotional recurrent unit for conversational sentiment analysis.
  arXiv  Detail & Related papers  (2020-05-31T11:13:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.