SemEval-2023 Task 10: Explainable Detection of Online Sexism

• URL: http://arxiv.org/abs/2303.04222v2
• Date: Mon, 8 May 2023 14:34:49 GMT
• Title: SemEval-2023 Task 10: Explainable Detection of Online Sexism
• Authors: Hannah Rose Kirk, Wenjie Yin, Bertie Vidgen, Paul R\"ottger
• Abstract summary: We introduce SemEval Task 10 on the Explainable Detection of Online Sexism (EDOS) We make three main contributions: i) a novel hierarchical taxonomy of sexist content, which includes granular vectors of sexism to aid explainability; ii) a new dataset of 20,000 social media comments with fine-grained labels, along with larger unlabelled datasets for model adaptation; andiii) baseline models as well as an analysis of the methods, results and errors for participant submissions to our task.
• Score: 5.542286527528687
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online sexism is a widespread and harmful phenomenon. Automated tools can assist the detection of sexism at scale. Binary detection, however, disregards the diversity of sexist content, and fails to provide clear explanations for why something is sexist. To address this issue, we introduce SemEval Task 10 on the Explainable Detection of Online Sexism (EDOS). We make three main contributions: i) a novel hierarchical taxonomy of sexist content, which includes granular vectors of sexism to aid explainability; ii) a new dataset of 20,000 social media comments with fine-grained labels, along with larger unlabelled datasets for model adaptation; and iii) baseline models as well as an analysis of the methods, results and errors for participant submissions to our task.

Related papers

• GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing [72.0343083866144]
  This paper introduces the GenderBias-emphVL benchmark to evaluate occupation-related gender bias in Large Vision-Language Models. Using our benchmark, we extensively evaluate 15 commonly used open-source LVLMs and state-of-the-art commercial APIs. Our findings reveal widespread gender biases in existing LVLMs.
  arXiv  Detail & Related papers  (2024-06-30T05:55:15Z)
• A multitask learning framework for leveraging subjectivity of annotators to identify misogyny [47.175010006458436]
  We propose a multitask learning approach to enhance the performance of the misogyny identification systems. We incorporated diverse perspectives from annotators in our model design, considering gender and age across six profile groups. This research advances content moderation and highlights the importance of embracing diverse perspectives to build effective online moderation systems.
  arXiv  Detail & Related papers  (2024-06-22T15:06:08Z)
• VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution [80.57383975987676]
  VisoGender is a novel dataset for benchmarking gender bias in vision-language models. We focus on occupation-related biases within a hegemonic system of binary gender, inspired by Winograd and Winogender schemas. We benchmark several state-of-the-art vision-language models and find that they demonstrate bias in resolving binary gender in complex scenes.
  arXiv  Detail & Related papers  (2023-06-21T17:59:51Z)
• LCT-1 at SemEval-2023 Task 10: Pre-training and Multi-task Learning for Sexism Detection and Classification [0.0]
  SemEval-2023 Task 10 on Explainable Detection of Online Sexism aims at increasing explainability of the sexism detection. Our system is based on further domain-adaptive pre-training. In experiments, multi-task learning performs on par with standard fine-tuning for sexism detection.
  arXiv  Detail & Related papers  (2023-06-08T09:56:57Z)
• Gender Biases in Automatic Evaluation Metrics for Image Captioning [87.15170977240643]
  We conduct a systematic study of gender biases in model-based evaluation metrics for image captioning tasks. We demonstrate the negative consequences of using these biased metrics, including the inability to differentiate between biased and unbiased generations. We present a simple and effective way to mitigate the metric bias without hurting the correlations with human judgments.
  arXiv  Detail & Related papers  (2023-05-24T04:27:40Z)
• AdamR at SemEval-2023 Task 10: Solving the Class Imbalance Problem in Sexism Detection with Ensemble Learning [1.9833664312284154]
  The Explainable Detection of Online Sexism task presents the problem of explainable sexism detection through fine-grained categorisation of sexist cases. Our team experimented with different ways to combat class imbalance throughout the tasks using data augmentation and loss alteration techniques. This solution ranked us in the top 40% of teams for each of the tracks.
  arXiv  Detail & Related papers  (2023-05-15T13:28:59Z)
• SSS at SemEval-2023 Task 10: Explainable Detection of Online Sexism using Majority Voted Fine-Tuned Transformers [0.0]
  This paper describes our submission to Task 10 at SemEval 2023-Explainable Detection of Online Sexism (EDOS) The recent rise in social media platforms has seen an increase in disproportionate levels of sexism experienced by women on social media platforms. Our approach consists of experimenting and finetuning BERT-based models and using a Majority Voting ensemble model that outperforms individual baseline model scores.
  arXiv  Detail & Related papers  (2023-04-07T07:24:32Z)
• Auditing Gender Presentation Differences in Text-to-Image Models [54.16959473093973]
  We study how gender is presented differently in text-to-image models. By probing gender indicators in the input text, we quantify the frequency differences of presentation-centric attributes. We propose an automatic method to estimate such differences.
  arXiv  Detail & Related papers  (2023-02-07T18:52:22Z)
• SWSR: A Chinese Dataset and Lexicon for Online Sexism Detection [9.443571652110663]
  We propose the first Chinese sexism dataset -- Sina Weibo Sexism Review (SWSR) dataset -- and a large Chinese lexicon SexHateLex. SWSR dataset provides labels at different levels of granularity including (i) sexism or non-sexism, (ii) sexism category and (iii) target type. We conduct experiments for the three sexism classification tasks making use of state-of-the-art machine learning models.
  arXiv  Detail & Related papers  (2021-08-06T12:06:40Z)
• Mitigating Gender Bias in Captioning Systems [56.25457065032423]
  Most captioning models learn gender bias, leading to high gender prediction errors, especially for women. We propose a new Guided Attention Image Captioning model (GAIC) which provides self-guidance on visual attention to encourage the model to capture correct gender visual evidence.
  arXiv  Detail & Related papers  (2020-06-15T12:16:19Z)
• "Call me sexist, but...": Revisiting Sexism Detection Using Psychological Scales and Adversarial Samples [2.029924828197095]
  We outline the different dimensions of sexism by grounding them in their implementation in psychological scales. From the scales, we derive a codebook for sexism in social media, which we use to annotate existing and novel datasets. Results indicate that current machine learning models pick up on a very narrow set of linguistic markers of sexism and do not generalize well to out-of-domain examples.
  arXiv  Detail & Related papers  (2020-04-27T13:07:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.