HarmPot: An Annotation Framework for Evaluating Offline Harm Potential of Social Media Text

• URL: http://arxiv.org/abs/2403.11108v1
• Date: Sun, 17 Mar 2024 06:23:25 GMT
• Title: HarmPot: An Annotation Framework for Evaluating Offline Harm Potential of Social Media Text
• Authors: Ritesh Kumar, Ojaswee Bhalla, Madhu Vanthi, Shehlat Maknoon Wani, Siddharth Singh,
• Abstract summary: We define "harm potential" as the potential for an online public post to cause real-world physical harm (i.e., violence) In this paper, we discuss the development of a framework/annotation schema that allows annotating the data with different aspects of the text.
• Score: 1.304892050913381
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we discuss the development of an annotation schema to build datasets for evaluating the offline harm potential of social media texts. We define "harm potential" as the potential for an online public post to cause real-world physical harm (i.e., violence). Understanding that real-world violence is often spurred by a web of triggers, often combining several online tactics and pre-existing intersectional fissures in the social milieu, to result in targeted physical violence, we do not focus on any single divisive aspect (i.e., caste, gender, religion, or other identities of the victim and perpetrators) nor do we focus on just hate speech or mis/dis-information. Rather, our understanding of the intersectional causes of such triggers focuses our attempt at measuring the harm potential of online content, irrespective of whether it is hateful or not. In this paper, we discuss the development of a framework/annotation schema that allows annotating the data with different aspects of the text including its socio-political grounding and intent of the speaker (as expressed through mood and modality) that together contribute to it being a trigger for offline harm. We also give a comparative analysis and mapping of our framework with some of the existing frameworks.

Related papers

• A Taxonomy of Rater Disagreements: Surveying Challenges & Opportunities from the Perspective of Annotating Online Toxicity [15.23055494327071]
  Toxicity is an increasingly common and severe issue in online spaces. A rich line of machine learning research has focused on computationally detecting and mitigating online toxicity. Recent research has pointed out the importance of accounting for the subjective nature of this task.
  arXiv  Detail & Related papers  (2023-11-07T21:00:51Z)
• Understanding writing style in social media with a supervised contrastively pre-trained transformer [57.48690310135374]
  Online Social Networks serve as fertile ground for harmful behavior, ranging from hate speech to the dissemination of disinformation. We introduce the Style Transformer for Authorship Representations (STAR), trained on a large corpus derived from public sources of 4.5 x 106 authored texts. Using a support base of 8 documents of 512 tokens, we can discern authors from sets of up to 1616 authors with at least 80% accuracy.
  arXiv  Detail & Related papers  (2023-10-17T09:01:17Z)
• Topological Data Mapping of Online Hate Speech, Misinformation, and General Mental Health: A Large Language Model Based Study [6.803493330690884]
  Recent advances in machine learning and large language models have made such an analysis possible. In this study, we collected thousands of posts from carefully selected communities on the social media site Reddit. We performed various machine-learning classifications based on embeddings in order to understand the role of hate speech/misinformation in various communities.
  arXiv  Detail & Related papers  (2023-09-22T15:10:36Z)
• CoSyn: Detecting Implicit Hate Speech in Online Conversations Using a Context Synergized Hyperbolic Network [52.85130555886915]
  CoSyn is a context-synergized neural network that explicitly incorporates user- and conversational context for detecting implicit hate speech in online conversations. We show that CoSyn outperforms all our baselines in detecting implicit hate speech with absolute improvements in the range of 1.24% - 57.8%.
  arXiv  Detail & Related papers  (2023-03-02T17:30:43Z)
• Qualitative Analysis of a Graph Transformer Approach to Addressing Hate Speech: Adapting to Dynamically Changing Content [8.393770595114763]
  We offer a detailed qualitative analysis of this solution for hate speech detection in social networks. A key insight is that the focus on reasoning about the concept of context positions us well to be able to support multi-modal analysis of online posts. We conclude with a reflection on how the problem we are addressing relates especially well to the theme of dynamic change.
  arXiv  Detail & Related papers  (2023-01-25T23:32:32Z)
• Detecting and Understanding Harmful Memes: A Survey [48.135415967633676]
  We offer a comprehensive survey with a focus on harmful memes. One interesting finding is that many types of harmful memes are not really studied, e.g., such featuring self-harm and extremism. Another observation is that memes can propagate globally through repackaging in different languages and that they can also be multilingual.
  arXiv  Detail & Related papers  (2022-05-09T13:43:27Z)
• Beyond Plain Toxic: Detection of Inappropriate Statements on Flammable Topics for the Russian Language [76.58220021791955]
  We present two text collections labelled according to binary notion of inapropriateness and a multinomial notion of sensitive topic. To objectivise the notion of inappropriateness, we define it in a data-driven way though crowdsourcing.
  arXiv  Detail & Related papers  (2022-03-04T15:59:06Z)
• Fragments of the Past: Curating Peer Support with Perpetrators of Domestic Violence [88.37416552778178]
  We report on a ten-month study where we worked with six support workers and eighteen perpetrators in the design and deployment of Fragments of the Past. We share how crafting digitally-augmented artefacts - 'fragments' - of experiences of desisting from violence can translate messages for motivation and rapport between peers. These insights provide the basis for practical considerations for future network design with challenging populations.
  arXiv  Detail & Related papers  (2021-07-09T22:57:43Z)
• Detecting Harmful Content On Online Platforms: What Platforms Need Vs. Where Research Efforts Go [44.774035806004214]
  harmful content on online platforms comes in many different forms including hate speech, offensive language, bullying and harassment, misinformation, spam, violence, graphic content, sexual abuse, self harm, and many other. Online platforms seek to moderate such content to limit societal harm, to comply with legislation, and to create a more inclusive environment for their users. There is currently a dichotomy between what types of harmful content online platforms seek to curb, and what research efforts there are to automatically detect such content.
  arXiv  Detail & Related papers  (2021-02-27T08:01:10Z)
• On Analyzing Annotation Consistency in Online Abusive Behavior Datasets [5.900114841365645]
  Researchers have proposed, collected, and annotated online abusive content datasets. These datasets play a critical role in facilitating the research on online hate speech and abusive behaviors. It is often contentious on what should be the true label of a given text as the semantic difference of the labels may be blurred.
  arXiv  Detail & Related papers  (2020-06-24T06:34:25Z)
• Transfer Learning for Hate Speech Detection in Social Media [14.759208309842178]
  This paper uses a transfer learning technique to leverage two independent datasets jointly. We build an interpretable two-dimensional visualization tool of the constructed hate speech representation -- dubbed the Map of Hate. We show that the joint representation boosts prediction performances when only a limited amount of supervision is available.
  arXiv  Detail & Related papers  (2019-06-10T08:00:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.