BioSentinel at EXIST 2026: Soft-Label Optimization with XLM-RoBERTa for Sexism Intent Classification in Memes

• URL: http://arxiv.org/abs/2607.24137v1
• Date: Mon, 27 Jul 2026 08:18:05 GMT
• Title: BioSentinel at EXIST 2026: Soft-Label Optimization with XLM-RoBERTa for Sexism Intent Classification in Memes
• Abstract summary: This paper describes the BioSentinel team's participation in EXIST 2026 Task 2.2: Source Intention in Memes.<n>The task requires classifying the communicative intent behind memes as direct, judgemental, or no (non-sexist)<n>We present a text-centric approach built on xlm-roberta-base (270M parameters) trained with a composite loss function combining KL divergence on soft annotator distributions.
• Score: 0.8749675983608171
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper describes the BioSentinel team's participation in EXIST 2026 Task 2.2: Source Intention in Memes, part of the CLEF 2026 evaluation campaign. The task requires classifying the communicative intent behind memes as direct, judgemental, or no (non-sexist), under a Learning with Disagreement (Le-Wi-Di) paradigm that mandates both hard-label and soft-label (probability distribution) predictions. We present a text-centric approach built on xlm-roberta-base (270M parameters) trained with a composite loss function combining KL divergence on soft annotator distributions and weighted cross-entropy on hard labels. On the official test set, the system achieved an ICM-Soft-Norm of 0.3229 and ICM-Norm of 0.3778, with a hard F1-score of 0.4236, ranking 40th (out of 118 submissions) in the soft-soft evaluation and 49th (out of 187 submissions) in the hard-hard evaluation. We provide an analysis of the dataset characteristics, exploratory larger-architecture runs, and the role of annotator disagreement in shaping model design for subjective NLP tasks. Ablation results show that KL loss improves soft-label metrics, while CE loss improves hard-label accuracy. We also report a separate validation-set temperature analysis.

Related papers

• Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence [0.7340017786387767]
  We show that prior scores, even when included only as context metadata, anchor judgments and systematically shift ratings toward their values.<n>On categorical industry data with human-labeled ground truth, anchored metadata blocks 48% of error corrections and flips 10.18% of correct judgments toward an assigned wrong label.
  arXiv  Detail & Related papers  (2026-08-26T14:41:24Z)
• Rethinking the Evaluation and Optimization of LLM-Based Social Simulation [56.18577997572273]
  We use the subjectivity coefficient to analyze how accuracy-based evaluation and hard-label training fail as subjectivity grows.<n>We construct SUBJSIM, a benchmark of 19,300 contexts covering 193 annotators and 100 subjective questions.<n>Results on SUBJSIM demonstrate the advantages of our method.
  arXiv  Detail & Related papers  (2026-08-20T06:26:03Z)
• Semantic Drift in Bug Resolution: How Behavioral Signals Propagate from Reports to Tests and Patches [18.727291516223115]
  Desc2Fix is a framework for measuring semantic alignment between bug reports, triggering tests, and developer-written fixes.<n>We analyze 2,857 report-test-patch triplets from Defects4J and SWT-Bench using GPT-4o and DeepSeek-Chat.
  arXiv  Detail & Related papers  (2026-07-20T22:32:23Z)
• Arena-T2I Hard: Benchmarking and Improving Faithfulness with Dependency-Aware Checklist [69.81453995715673]
  Existing faithfulness benchmarks rely on simple atomic instructions.<n>We introduce Arena-T2I Hard, a 310-prompt stress benchmark drawn from real arena T2I logs.<n>We propose a dependency-aware checklist reward that decomposes each prompt into a DAG of yes/no questions and zeroes descendants of failed parents.
  arXiv  Detail & Related papers  (2026-06-30T14:17:05Z)
• Auditing LLM Benchmarks with Item Response Theory [1.1455131351238124]
  LLM benchmark labels are frozen at release and silently propagated into downstream benchmarks, errors and all.<n>We introduce an Item Response Theory-based indicator that surfaces likely mislabels at 95% precision in the top 200 examples.<n>We trace these errors to mechanical labelings, upstream annotation mistakes inherited unchanged from source datasets, and fundamentally ambiguous items without a defensible single label.
  arXiv  Detail & Related papers  (2026-05-28T19:38:44Z)
• A Controlled Synthetic Benchmark for Educational Aspect-Based Sentiment Analysis [47.027290803102666]
  This study introduces a controlled benchmark for educational ABSA built from 10,000 synthetic course reviews.<n>The strongest untuned model, BERT, reaches a held-out detection micro-F1 of 0.2760.<n>A conservative external evaluation on 2,829 mapped student-feedback reviews from Herath et al. yields a micro-F1 of 0.4593 for BERT on a 9-aspect overlap.
  arXiv  Detail & Related papers  (2026-05-25T07:05:21Z)
• STaR-DRO: Stateful Tsallis Reweighting for Group-Robust Structured Prediction [0.20165644958637055]
  We present a two-part framework for controllable inference and robust fine-tuning.<n>First, we introduce a task-agnostic prompting strategy to address format drift, label ambiguity, evidence hallucination, and metadata-conditioned confusion.<n>Second, we introduce STaR-DRO, a stateful robust optimization method for group heterogeneity.
  arXiv  Detail & Related papers  (2026-04-09T21:58:42Z)
• Distributional Open-Ended Evaluation of LLM Cultural Value Alignment Based on Value Codebook [50.10594064510559]
  DOVE is a distributional evaluation framework that directly compares human-written text with LLM-generated outputs.<n>We show that DOVE achieves superior predictive validity, attaining a 31.56% correlation with downstream tasks, while maintaining high reliability with as few as 500 samples per culture.
  arXiv  Detail & Related papers  (2026-03-16T08:33:10Z)
• Loss Design and Architecture Selection for Long-Tailed Multi-Label Chest X-Ray Classification [0.0]
  Longtailed distributions class pose a significant challenge for multi-label chest X-ray classification.<n>We present a systematic empirical evaluation of loss functions, CNN backbone architectures and post-training strategies on the CXR-LT 2026 benchmark.<n>Our experiments demonstrate that LDAM with deferred re-weighting consistently outperforms standard BCE and asymmetric losses for rare class recognition.
  arXiv  Detail & Related papers  (2026-03-02T17:33:00Z)
• Uncertainty-aware Long-tailed Weights Model the Utility of Pseudo-labels for Semi-supervised Learning [50.868594148443215]
  We propose an Uncertainty-aware Ensemble Structure (UES) to assess the utility of pseudo-labels for unlabeled samples.<n>UES is lightweight and architecture-agnostic, easily extending to various computer vision tasks, including classification and regression.
  arXiv  Detail & Related papers  (2025-03-13T02:21:04Z)
• Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets [92.38654521870444]
  We introduce ACES, a contrastive challenge set spanning 146 language pairs. This dataset aims to discover whether metrics can identify 68 translation accuracy errors. We conduct a large-scale study by benchmarking ACES on 50 metrics submitted to the WMT 2022 and 2023 metrics shared tasks.
  arXiv  Detail & Related papers  (2024-01-29T17:17:42Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.