Whose Opinions Matter? Perspective-aware Models to Identify Opinions of Hate Speech Victims in Abusive Language Detection

• URL: http://arxiv.org/abs/2106.15896v1
• Date: Wed, 30 Jun 2021 08:35:49 GMT
• Title: Whose Opinions Matter? Perspective-aware Models to Identify Opinions of Hate Speech Victims in Abusive Language Detection
• Authors: Sohail Akhtar, Valerio Basile, Viviana Patti
• Abstract summary: We present an in-depth study to model polarized opinions coming from different communities. We believe that by relying on this information, we can divide the annotators into groups sharing similar perspectives. We propose a novel resource, a multi-perspective English language dataset annotated according to different sub-categories relevant for characterising online abuse.
• Score: 6.167830237917662
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Social media platforms provide users the freedom of expression and a medium to exchange information and express diverse opinions. Unfortunately, this has also resulted in the growth of abusive content with the purpose of discriminating people and targeting the most vulnerable communities such as immigrants, LGBT, Muslims, Jews and women. Because abusive language is subjective in nature, there might be highly polarizing topics or events involved in the annotation of abusive contents such as hate speech (HS). Therefore, we need novel approaches to model conflicting perspectives and opinions coming from people with different personal and demographic backgrounds. In this paper, we present an in-depth study to model polarized opinions coming from different communities under the hypothesis that similar characteristics (ethnicity, social background, culture etc.) can influence the perspectives of annotators on a certain phenomenon. We believe that by relying on this information, we can divide the annotators into groups sharing similar perspectives. We can create separate gold standards, one for each group, to train state-of-the-art deep learning models. We can employ an ensemble approach to combine the perspective-aware classifiers from different groups to an inclusive model. We also propose a novel resource, a multi-perspective English language dataset annotated according to different sub-categories relevant for characterising online abuse: hate speech, aggressiveness, offensiveness and stereotype. By training state-of-the-art deep learning models on this novel resource, we show how our approach improves the prediction performance of a state-of-the-art supervised classifier.

Related papers

• Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models [50.40276881893513]
  This study introduces Spoken Stereoset, a dataset specifically designed to evaluate social biases in Speech Large Language Models (SLLMs) By examining how different models respond to speech from diverse demographic groups, we aim to identify these biases. The findings indicate that while most models show minimal bias, some still exhibit slightly stereotypical or anti-stereotypical tendencies.
  arXiv  Detail & Related papers  (2024-08-14T16:55:06Z)
• Protected group bias and stereotypes in Large Language Models [2.1122940074160357]
  This paper investigates the behavior of Large Language Models (LLMs) in the domains of ethics and fairness. We find bias across minoritized groups, but in particular in the domains of gender and sexuality, as well as Western bias.
  arXiv  Detail & Related papers  (2024-03-21T00:21:38Z)
• On the steerability of large language models toward data-driven personas [98.9138902560793]
  Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented. Here, we present a novel approach to achieve controllable generation of specific viewpoints using LLMs.
  arXiv  Detail & Related papers  (2023-11-08T19:01:13Z)
• Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems [103.416202777731]
  We study "persona biases", which we define to be the sensitivity of dialogue models' harmful behaviors contingent upon the personas they adopt. We categorize persona biases into biases in harmful expression and harmful agreement, and establish a comprehensive evaluation framework to measure persona biases in five aspects: Offensiveness, Toxic Continuation, Regard, Stereotype Agreement, and Toxic Agreement.
  arXiv  Detail & Related papers  (2023-10-08T21:03:18Z)
• Foundational Models Defining a New Era in Vision: A Survey and Outlook [151.49434496615427]
  Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world. The models learned to bridge the gap between such modalities coupled with large-scale training data facilitate contextual reasoning, generalization, and prompt capabilities at test time. The output of such models can be modified through human-provided prompts without retraining, e.g., segmenting a particular object by providing a bounding box, having interactive dialogues by asking questions about an image or video scene or manipulating the robot's behavior through language instructions.
  arXiv  Detail & Related papers  (2023-07-25T17:59:18Z)
• Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale [61.555788332182395]
  We investigate the potential for machine learning models to amplify dangerous and complex stereotypes. We find a broad range of ordinary prompts produce stereotypes, including prompts simply mentioning traits, descriptors, occupations, or objects.
  arXiv  Detail & Related papers  (2022-11-07T18:31:07Z)
• Revealing Persona Biases in Dialogue Systems [64.96908171646808]
  We present the first large-scale study on persona biases in dialogue systems. We conduct analyses on personas of different social classes, sexual orientations, races, and genders. In our studies of the Blender and DialoGPT dialogue systems, we show that the choice of personas can affect the degree of harms in generated responses.
  arXiv  Detail & Related papers  (2021-04-18T05:44:41Z)
• Interpretable Multi-Modal Hate Speech Detection [32.36781061930129]
  We propose a deep neural multi-modal model that can effectively capture the semantics of the text along with socio-cultural context in which a particular hate expression is made. Our model is able to outperform the existing state-of-the-art hate speech classification approaches.
  arXiv  Detail & Related papers  (2021-03-02T10:12:26Z)
• Multilingual Contextual Affective Analysis of LGBT People Portrayals in Wikipedia [34.183132688084534]
  Specific lexical choices in narrative text reflect both the writer's attitudes towards people in the narrative and influence the audience's reactions. We show how word connotations differ across languages and cultures, highlighting the difficulty of generalizing existing English datasets. We then demonstrate the usefulness of our method by analyzing Wikipedia biography pages of members of the LGBT community across three languages.
  arXiv  Detail & Related papers  (2020-10-21T08:27:36Z)
• Examining Racial Bias in an Online Abuse Corpus with Structural Topic Modeling [0.30458514384586405]
  We use structural topic modeling to examine racial bias in social media posts. We augment the abusive language dataset by adding an additional feature indicating the predicted probability of the tweet being written in African-American English.
  arXiv  Detail & Related papers  (2020-05-26T21:02:43Z)
• Transfer Learning for Hate Speech Detection in Social Media [14.759208309842178]
  This paper uses a transfer learning technique to leverage two independent datasets jointly. We build an interpretable two-dimensional visualization tool of the constructed hate speech representation -- dubbed the Map of Hate. We show that the joint representation boosts prediction performances when only a limited amount of supervision is available.
  arXiv  Detail & Related papers  (2019-06-10T08:00:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.