COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements

• URL: http://arxiv.org/abs/2306.01985v2
• Date: Fri, 9 Jun 2023 01:49:06 GMT
• Title: COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements
• Authors: Xuhui Zhou, Hao Zhu, Akhila Yerukola, and Thomas Davidson, Jena D. Hwang, Swabha Swayamdipta, Maarten Sap
• Abstract summary: We introduce COBRA frames, the first context-aware formalism for explaining the intents, reactions, and harms of offensive or biased statements. We create COBRACORPUS, a dataset of 33k potentially offensive statements paired with machine-generated contexts. We find that explanations by context-agnostic models are significantly worse than by context-aware ones.
• Score: 30.1056760312051
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Warning: This paper contains content that may be offensive or upsetting. Understanding the harms and offensiveness of statements requires reasoning about the social and situational context in which statements are made. For example, the utterance "your English is very good" may implicitly signal an insult when uttered by a white man to a non-white colleague, but uttered by an ESL teacher to their student would be interpreted as a genuine compliment. Such contextual factors have been largely ignored by previous approaches to toxic language detection. We introduce COBRA frames, the first context-aware formalism for explaining the intents, reactions, and harms of offensive or biased statements grounded in their social and situational context. We create COBRACORPUS, a dataset of 33k potentially offensive statements paired with machine-generated contexts and free-text explanations of offensiveness, implied biases, speaker intents, and listener reactions. To study the contextual dynamics of offensiveness, we train models to generate COBRA explanations, with and without access to the context. We find that explanations by context-agnostic models are significantly worse than by context-aware ones, especially in situations where the context inverts the statement's offensiveness (29% accuracy drop). Our work highlights the importance and feasibility of contextualized NLP by modeling social factors.

Related papers

• A Comprehensive View of the Biases of Toxicity and Sentiment Analysis Methods Towards Utterances with African American English Expressions [5.472714002128254]
  We study bias on two Web-based (YouTube and Twitter) datasets and two spoken English datasets. We isolate the impact of AAE expression usage via linguistic control features from the Linguistic Inquiry and Word Count software. We present consistent results on how a heavy usage of AAE expressions may cause the speaker to be considered substantially more toxic, even when speaking about nearly the same subject.
  arXiv  Detail & Related papers  (2024-01-23T12:41:03Z)
• Natural Language Decompositions of Implicit Content Enable Better Text Representations [56.85319224208865]
  We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account. We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed. Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
  arXiv  Detail & Related papers  (2023-05-23T23:45:20Z)
• Hate Speech and Counter Speech Detection: Conversational Context Does Matter [7.333666276087548]
  This paper investigates the role of conversational context in the annotation and detection of online hate and counter speech. We created a context-aware dataset for a 3-way classification task on Reddit comments: hate speech, counter speech, or neutral.
  arXiv  Detail & Related papers  (2022-06-13T19:05:44Z)
• Rethinking Offensive Text Detection as a Multi-Hop Reasoning Problem [15.476899850339395]
  We introduce the task of implicit offensive text detection in dialogues. We argue that reasoning is crucial for understanding this broader class of offensive utterances. We release SLIGHT, a dataset to support research on this task.
  arXiv  Detail & Related papers  (2022-04-22T06:20:15Z)
• Beyond Plain Toxic: Detection of Inappropriate Statements on Flammable Topics for the Russian Language [76.58220021791955]
  We present two text collections labelled according to binary notion of inapropriateness and a multinomial notion of sensitive topic. To objectivise the notion of inappropriateness, we define it in a data-driven way though crowdsourcing.
  arXiv  Detail & Related papers  (2022-03-04T15:59:06Z)
• Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts [26.660268192685763]
  We crowd-annotate ToxiChat, a new dataset of 2,000 Reddit threads and model responses labeled with offensive language and stance. Our analysis reveals that 42% of user responses agree with toxic comments; 3x their agreement with safe comments.
  arXiv  Detail & Related papers  (2021-08-26T14:58:05Z)
• Do Context-Aware Translation Models Pay the Right Attention? [61.25804242929533]
  Context-aware machine translation models are designed to leverage contextual information, but often fail to do so. In this paper, we ask several questions: What contexts do human translators use to resolve ambiguous words? We introduce SCAT (Supporting Context for Ambiguous Translations), a new English-French dataset comprising supporting context words for 14K translations. Using SCAT, we perform an in-depth analysis of the context used to disambiguate, examining positional and lexical characteristics of the supporting words.
  arXiv  Detail & Related papers  (2021-05-14T17:32:24Z)
• Identifying Offensive Expressions of Opinion in Context [0.0]
  It is still a challenge to subjective information extraction systems to identify opinions and feelings in context. In sentiment-based NLP tasks, there are few resources to information extraction, above all offensive or hateful opinions in context. This paper provides a new cross-lingual and contextual offensive lexicon, which consists of explicit and implicit offensive and swearing expressions of opinion.
  arXiv  Detail & Related papers  (2021-04-25T18:35:39Z)
• Did they answer? Subjective acts and intents in conversational discourse [48.63528550837949]
  We present the first discourse dataset with multiple and subjective interpretations of English conversation. We show disagreements are nuanced and require a deeper understanding of the different contextual factors.
  arXiv  Detail & Related papers  (2021-04-09T16:34:19Z)
• A Deep Neural Framework for Contextual Affect Detection [51.378225388679425]
  A short and simple text carrying no emotion can represent some strong emotions when reading along with its context. We propose a Contextual Affect Detection framework which learns the inter-dependence of words in a sentence.
  arXiv  Detail & Related papers  (2020-01-28T05:03:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.