"Call me sexist, but...": Revisiting Sexism Detection Using Psychological Scales and Adversarial Samples

• URL: http://arxiv.org/abs/2004.12764v2
• Date: Wed, 2 Jun 2021 10:39:03 GMT
• Title: "Call me sexist, but...": Revisiting Sexism Detection Using Psychological Scales and Adversarial Samples
• Authors: Mattia Samory, Indira Sen, Julian Kohne, Fabian Floeck, Claudia Wagner
• Abstract summary: We outline the different dimensions of sexism by grounding them in their implementation in psychological scales. From the scales, we derive a codebook for sexism in social media, which we use to annotate existing and novel datasets. Results indicate that current machine learning models pick up on a very narrow set of linguistic markers of sexism and do not generalize well to out-of-domain examples.
• Score: 2.029924828197095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Research has focused on automated methods to effectively detect sexism online. Although overt sexism seems easy to spot, its subtle forms and manifold expressions are not. In this paper, we outline the different dimensions of sexism by grounding them in their implementation in psychological scales. From the scales, we derive a codebook for sexism in social media, which we use to annotate existing and novel datasets, surfacing their limitations in breadth and validity with respect to the construct of sexism. Next, we leverage the annotated datasets to generate adversarial examples, and test the reliability of sexism detection methods. Results indicate that current machine learning models pick up on a very narrow set of linguistic markers of sexism and do not generalize well to out-of-domain examples. Yet, including diverse data and adversarial examples at training time results in models that generalize better and that are more robust to artifacts of data collection. By providing a scale-based codebook and insights regarding the shortcomings of the state-of-the-art, we hope to contribute to the development of better and broader models for sexism detection, including reflections on theory-driven approaches to data collection.

Related papers

• A multitask learning framework for leveraging subjectivity of annotators to identify misogyny [47.175010006458436]
  We propose a multitask learning approach to enhance the performance of the misogyny identification systems. We incorporated diverse perspectives from annotators in our model design, considering gender and age across six profile groups. This research advances content moderation and highlights the importance of embracing diverse perspectives to build effective online moderation systems.
  arXiv  Detail & Related papers  (2024-06-22T15:06:08Z)
• LCT-1 at SemEval-2023 Task 10: Pre-training and Multi-task Learning for Sexism Detection and Classification [0.0]
  SemEval-2023 Task 10 on Explainable Detection of Online Sexism aims at increasing explainability of the sexism detection. Our system is based on further domain-adaptive pre-training. In experiments, multi-task learning performs on par with standard fine-tuning for sexism detection.
  arXiv  Detail & Related papers  (2023-06-08T09:56:57Z)
• Gender Biases in Automatic Evaluation Metrics for Image Captioning [87.15170977240643]
  We conduct a systematic study of gender biases in model-based evaluation metrics for image captioning tasks. We demonstrate the negative consequences of using these biased metrics, including the inability to differentiate between biased and unbiased generations. We present a simple and effective way to mitigate the metric bias without hurting the correlations with human judgments.
  arXiv  Detail & Related papers  (2023-05-24T04:27:40Z)
• SemEval-2023 Task 10: Explainable Detection of Online Sexism [5.542286527528687]
  We introduce SemEval Task 10 on the Explainable Detection of Online Sexism (EDOS) We make three main contributions: i) a novel hierarchical taxonomy of sexist content, which includes granular vectors of sexism to aid explainability; ii) a new dataset of 20,000 social media comments with fine-grained labels, along with larger unlabelled datasets for model adaptation; andiii) baseline models as well as an analysis of the methods, results and errors for participant submissions to our task.
  arXiv  Detail & Related papers  (2023-03-07T20:28:39Z)
• Auditing Gender Presentation Differences in Text-to-Image Models [54.16959473093973]
  We study how gender is presented differently in text-to-image models. By probing gender indicators in the input text, we quantify the frequency differences of presentation-centric attributes. We propose an automatic method to estimate such differences.
  arXiv  Detail & Related papers  (2023-02-07T18:52:22Z)
• SexWEs: Domain-Aware Word Embeddings via Cross-lingual Semantic Specialisation for Chinese Sexism Detection in Social Media [23.246615034191553]
  We develop a cross-lingual domain-aware semantic specialisation system for sexism detection. We leverage semantic resources for sexism from a high-resource language (English) to specialise pre-trained word vectors in the target language (Chinese) to inject domain knowledge. Compared with other specialisation approaches and Chinese baseline word vectors, our SexWEs shows an average score improvement of 0.033 and 0.064 in both intrinsic and extrinsic evaluations.
  arXiv  Detail & Related papers  (2022-11-15T19:00:20Z)
• DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models [73.12069620086311]
  We investigate the visual reasoning capabilities and social biases of text-to-image models. First, we measure three visual reasoning skills: object recognition, object counting, and spatial relation understanding. Second, we assess the gender and skin tone biases by measuring the gender/skin tone distribution of generated images.
  arXiv  Detail & Related papers  (2022-02-08T18:36:52Z)
• SWSR: A Chinese Dataset and Lexicon for Online Sexism Detection [9.443571652110663]
  We propose the first Chinese sexism dataset -- Sina Weibo Sexism Review (SWSR) dataset -- and a large Chinese lexicon SexHateLex. SWSR dataset provides labels at different levels of granularity including (i) sexism or non-sexism, (ii) sexism category and (iii) target type. We conduct experiments for the three sexism classification tasks making use of state-of-the-art machine learning models.
  arXiv  Detail & Related papers  (2021-08-06T12:06:40Z)
• Automatic Detection of Sexist Statements Commonly Used at the Workplace [0.9790236766474201]
  We present a dataset of sexist statements that are more likely to be said in the workplace. We also present a deep learning model that can achieve state-of-the art results.
  arXiv  Detail & Related papers  (2020-07-08T15:14:29Z)
• Mitigating Gender Bias in Captioning Systems [56.25457065032423]
  Most captioning models learn gender bias, leading to high gender prediction errors, especially for women. We propose a new Guided Attention Image Captioning model (GAIC) which provides self-guidance on visual attention to encourage the model to capture correct gender visual evidence.
  arXiv  Detail & Related papers  (2020-06-15T12:16:19Z)
• Multi-Dimensional Gender Bias Classification [67.65551687580552]
  Machine learning models can inadvertently learn socially undesirable patterns when training on gender biased text. We propose a general framework that decomposes gender bias in text along several pragmatic and semantic dimensions. Using this fine-grained framework, we automatically annotate eight large scale datasets with gender information.
  arXiv  Detail & Related papers  (2020-05-01T21:23:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.