Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions

• URL: http://arxiv.org/abs/2604.20168v1
• Date: Wed, 22 Apr 2026 04:18:14 GMT
• Title: Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions
• Abstract summary: We address Task 1 (clarity-level classification) and Task 2 (evasion-level classification)<n>Our system is based on DeBERTa-V3-base, extended with focal loss, layer-wise learning rate decay, and Duluth discourse features.<n>To address class imbalance in the training data, we augment minority classes using synthetic examples generated by Gemini 3 and Claude Sonnet 4.5.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents the Duluth approach to SemEval-2026 Task 6 on CLARITY: Unmasking Political Question Evasions. We address Task 1 (clarity-level classification) and Task 2 (evasion-level classification), both of which involve classifying question--answer pairs from U.S.\ presidential interviews using a two-level taxonomy of response clarity. Our system is based on DeBERTa-V3-base, extended with focal loss, layer-wise learning rate decay, and boolean discourse features. To address class imbalance in the training data, we augment minority classes using synthetic examples generated by Gemini 3 and Claude Sonnet 4.5. Our best configuration achieved a Macro F1 of 0.76 on the Task 1 evaluation set, placing 8th out of 40 teams. The top-ranked system (TeleAI) achieved 0.89, while the mean score across participants was 0.70. Error analysis reveals that the dominant source of misclassification is confusion between Ambivalent and Clear Reply responses, a pattern that mirrors disagreements among human annotators. Our findings demonstrate that LLM-based data augmentation can meaningfully improve minority-class recall on nuanced political discourse tasks.

Related papers

• PolERo: Studying Political Evasion in Romanian [0.6445605125467574]
  Political evasion refers to responses that engage with a question while withholding the requested information.<n>Recent NLP work frames political evasion as a classification task using a two-level taxonomy of response clarity and fine-grained evasion strategies.<n>We introduce PolERo, a dataset of 3,574 human-annotated question-answer pairs extracted from official transcripts of five Romanian presidents.
  arXiv  Detail & Related papers  (2026-09-02T10:02:57Z)
• ttda704 at SemEval-2026 Task 6: Structured Chain-of-Thought Prompting for Political Evasion Detection [0.006227637802384129]
  This paper describes our system for SemEval-2026 Task 6, which addresses the classification of political evasion strategies in English question-answer pairs extracted from U.S. presidential interviews.
  arXiv  Detail & Related papers  (2026-06-14T12:09:23Z)
• Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection [68.37351671559675]
  Active learning can annotate thousands of instances from a short prompt at negligible cost.<n>This raises two questions for active learning (AL): can LLM labels replace human labels within the AL loop, and does AL remain necessary when entire corpora can be labelled at once?<n>We investigate both questions on a new dataset of 277,902 German political TikTok comments.
  arXiv  Detail & Related papers  (2026-04-15T14:10:58Z)
• SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions [17.397909377851366]
  Political speakers often avoid answering questions directly while maintaining the appearance of responsiveness.<n>We introduce SemEval-2026 Task 6, CLARITY, a shared task on political question evasion.<n>The benchmark is constructed from U.S. presidential interviews and follows an expert-grounded taxonomy of response clarity and evasion.
  arXiv  Detail & Related papers  (2026-03-14T17:02:52Z)
• UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning [101.62386137855704]
  We present a novel Universal Multimodal Embedding (UniME-V2) model.<n>Our approach first constructs a potential hard negative set through global retrieval.<n>We then introduce the MLLM-as-a-Judge mechanism, which utilizes MLLMs to assess the semantic alignment of query-candidate pairs.<n>These scores serve as a foundation for hard negative mining, mitigating the impact of false negatives and enabling the identification of diverse, high-quality hard negatives.
  arXiv  Detail & Related papers  (2025-10-15T13:07:00Z)
• Targeting Alignment: Extracting Safety Classifiers of Aligned LLMs [6.462219916993885]
  Alignment in large language models (LLMs) is used to enforce guidelines such as safety.<n>Yet, alignment fails in the face of jailbreak attacks that modify inputs to induce unsafe outputs.<n>We introduce and evaluate a new technique for jailbreak attacks.
  arXiv  Detail & Related papers  (2025-01-27T22:13:05Z)
• LLM Robustness Against Misinformation in Biomedical Question Answering [50.98256373698759]
  The retrieval-augmented generation (RAG) approach is used to reduce the confabulation of large language models (LLMs) for question answering. We evaluate the effectiveness and robustness of four LLMs against misinformation in answering biomedical questions.
  arXiv  Detail & Related papers  (2024-10-27T16:23:26Z)
• MACAROON: Training Vision-Language Models To Be Your Engaged Partners [95.32771929749514]
  Large vision-language models (LVLMs) generate detailed responses even when questions are ambiguous or unlabeled. In this study, we aim to shift LVLMs from passive answer providers to proactive engaged partners. We introduce MACAROON, self-iMaginAtion for ContrAstive pReference OptimizatiON, which instructs LVLMs to autonomously generate contrastive response pairs for unlabeled questions.
  arXiv  Detail & Related papers  (2024-06-20T09:27:33Z)
• Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values [73.82043713141142]
  Many NLP classification tasks, such as sexism/racism detection or toxicity detection, are based on human values. We introduce a framework for value-aligned classification that performs prediction based on explicitly written human values in the command.
  arXiv  Detail & Related papers  (2022-10-14T09:10:49Z)
• Deep F-measure Maximization for End-to-End Speech Understanding [52.36496114728355]
  We propose a differentiable approximation to the F-measure and train the network with this objective using standard backpropagation. We perform experiments on two standard fairness datasets, Adult, Communities and Crime, and also on speech-to-intent detection on the ATIS dataset and speech-to-image concept classification on the Speech-COCO dataset. In all four of these tasks, F-measure results in improved micro-F1 scores, with absolute improvements of up to 8% absolute, as compared to models trained with the cross-entropy loss function.
  arXiv  Detail & Related papers  (2020-08-08T03:02:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.