Through the Eyes of the Beholder: Biometric and Demographic Conditioning for Multimodal Sexism Detection

• URL: http://arxiv.org/abs/2609.15608v1
• Date: Mon, 14 Sep 2026 14:05:13 GMT
• Title: Through the Eyes of the Beholder: Biometric and Demographic Conditioning for Multimodal Sexism Detection
• Abstract summary: We propose a human-centered framework that analyses and incorporates the psychological and demographic characteristics of human annotators into the detection pipeline.<n>Meme text is extracted and visually described with Gemma 4, then augmented by automatic translation between English and Spanish with NLLB-200.<n>At inference time, predictions are produced by soft-voting between the deep multimodal network and a complementary SVM trained on stylometric and physiological features.
• Score: 1.0165430007278446
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Detecting sexism on the internet is a fundamentally subjective task; our team, VANGUARD, addresses this challenge in the EXIST 2026 Task 2 by proposing a human-centered multimodal framework that analyses and incorporates the psychological and demographic characteristics of human annotators into the detection pipeline. We fuse five input modalities through a cross-attention architecture with Feature-wise Linear Modulation conditioning. Meme text is extracted and visually described with Gemma 4, then augmented by automatic translation between English and Spanish with NLLB-200. Text and image representations are produced by LoRAadapted XLM-RoBERTa and CLIP encoders and fused with sensor features encoded by a pretrained autoencoder. To model annotator subjectivity, we frame Subtask 2.1 as a label distribution learning problem, optimizing a Kullback-Leibler divergence loss over the full annotator label distribution. At inference time, predictions are produced by soft-voting between the deep multimodal network and a complementary SVM trained on stylometric and physiological features. Our best submission ranks 29th out of 114 on Subtask 2.2 (source intention) under soft evaluation, and the normalized ICM scores remain above the baseline on Subtasks 2.1 and 2.2, indicating that annotator-centered conditioning contributes a usable signal. We release our full pipeline and analysis to support reproducible human-centered modeling.

Related papers

• Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis [55.78822230833247]
  We present Inter-X++, a comprehensive and large-scale benchmark designed to empower versatile HHI analysis.<n>Inter-X++ provides 11,388 high-fidelity interaction sequences and over 8.1M frames, featuring precise whole-body movements and detailed finger articulations.<n>We propose OpenHHI, a single and unified HHI representation and modeling framework that jointly optimize interaction reconstruction and semantic understanding.
  arXiv  Detail & Related papers  (2026-08-20T17:51:48Z)
• UC-VLM: Consistency-Driven Learning for AI-Generated Image Detection with Vision-Language Large Models [34.80872382508184]
  UC-VLM is a unified multi-stage framework for AIGI detection that relies solely on binary supervision.<n>It consistently reuses the same authenticity labels for visual adaptation and label-conditioned text generation.<n>Experiments show that UC-VLM achieves 96.1% average accuracy on GenImage, exceeding the strongest prior result by 4.6%.
  arXiv  Detail & Related papers  (2026-08-15T13:53:59Z)
• Multimodal Sexism Identification and Characterization using Large Language Models and Gradient Boosting [5.234175699955086]
  We address multimodal sexism identification and characterization in memes and short-form videos.<n>For memes, we combine visual, textual, demographic, biometric, and LLM-derived semantic indicators.<n>For videos, we investigate the effect of feature selection, frame-based visual representations, OCR-based textual features, acoustic descriptors, and sensor-derived metadata.
  arXiv  Detail & Related papers  (2026-06-04T10:43:46Z)
• TempRet: Temporal Enhancement and Two-Stage Reranking for CVPR 2026 EPIC-KITCHENS-100 Multi-Instance Retrieval Challenge [71.10535279591527]
  We present our solution, termed TempRet, to the CVPR 2026 EPIC-KITCHENS-100 MIR challenge.<n>Our approach builds upon a CLIP-based dual-encoder backbone and introduces two key components to address the temporal and cross-modal challenges.<n>Our method achieves 67.97% average mAP and 82.92% average nDCG on the EK-100 MIR benchmark.
  arXiv  Detail & Related papers  (2026-05-23T08:37:39Z)
• Why AI-Generated Text Detection Fails: Evidence from Explainable AI Beyond Benchmark Accuracy [0.9169660430821364]
  We propose a framework that integrates linguistic feature engineering, machine learning, and explainable AI techniques.<n>Using SHAP-based explanations, we show that the most influential features differ markedly between datasets.<n>We believe that this knowledge helps build AI detectors that are robust across different settings.
  arXiv  Detail & Related papers  (2026-03-24T12:46:15Z)
• Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection [59.04089915447622]
  ForenAgent is an interactive IFD framework that enables MLLMs to autonomously generate, execute, and refine Python-based low-level tools around the detection objective.<n>Inspired by human reasoning, we design a dynamic reasoning loop comprising global perception, local focusing, iterative probing, and holistic adjudication.<n>Experiments show that ForenAgent exhibits emergent tool-use competence and reflective reasoning on challenging IFD tasks.
  arXiv  Detail & Related papers  (2025-12-18T08:38:44Z)
• GAME: Learning Multimodal Interactions via Graph Structures for Personality Trait Estimation [13.071227081328288]
  Apparent personality analysis from short videos poses significant chal-lenges due to the complex interplay of visual, auditory, and textual cues.<n>In this paper, we propose GAME, a Graph-Augmented Multimodalvolution are designed to robustly model and fuse multi-source features for automatic personality prediction.<n>For the visual stream, we construct a facial graph and introduce a dual-branch Geo Two-Stream Network, which combines Graph Convolutional Networks (GCNs) and Convolutional Neural Net-works (CNNs)<n>To capture temporal dynamics, frame-level features are processed by a BiG
  arXiv  Detail & Related papers  (2025-05-05T13:48:09Z)
• The MuSe 2024 Multimodal Sentiment Analysis Challenge: Social Perception and Humor Recognition [64.5207572897806]
  The Multimodal Sentiment Analysis Challenge (MuSe) 2024 addresses two contemporary multimodal affect and sentiment analysis problems. In the Social Perception Sub-Challenge (MuSe-Perception), participants will predict 16 different social attributes of individuals. The Cross-Cultural Humor Detection Sub-Challenge (MuSe-Humor) dataset expands upon the Passau Spontaneous Football Coach Humor dataset.
  arXiv  Detail & Related papers  (2024-06-11T22:26:20Z)
• Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
  Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly. Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness. Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings. This paper introduces an innovative multi-modal prompt-based methodology for IQA.
  arXiv  Detail & Related papers  (2024-04-23T11:45:32Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• Object Segmentation by Mining Cross-Modal Semantics [68.88086621181628]
  We propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features. Specifically, we propose a novel network, termed XMSNet, consisting of (1) all-round attentive fusion (AF), (2) coarse-to-fine decoder (CFD), and (3) cross-layer self-supervision.
  arXiv  Detail & Related papers  (2023-05-17T14:30:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.