SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions

• URL: http://arxiv.org/abs/2603.14027v1
• Date: Sat, 14 Mar 2026 17:02:52 GMT
• Title: SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions
• Abstract summary: Political speakers often avoid answering questions directly while maintaining the appearance of responsiveness.<n>We introduce SemEval-2026 Task 6, CLARITY, a shared task on political question evasion.<n>The benchmark is constructed from U.S. presidential interviews and follows an expert-grounded taxonomy of response clarity and evasion.
• Score: 17.397909377851366
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Political speakers often avoid answering questions directly while maintaining the appearance of responsiveness. Despite its importance for public discourse, such strategic evasion remains underexplored in Natural Language Processing. We introduce SemEval-2026 Task 6, CLARITY, a shared task on political question evasion consisting of two subtasks: (i) clarity-level classification into Clear Reply, Ambivalent, and Clear Non-Reply, and (ii) evasion-level classification into nine fine-grained evasion strategies. The benchmark is constructed from U.S. presidential interviews and follows an expert-grounded taxonomy of response clarity and evasion. The task attracted 124 registered teams, who submitted 946 valid runs for clarity-level classification and 539 for evasion-level classification. Results show a substantial gap in difficulty between the two subtasks: the best system achieved 0.89 macro-F1 on clarity classification, surpassing the strongest baseline by a large margin, while the top evasion-level system reached 0.68 macro-F1, matching the best baseline. Overall, large language model prompting and hierarchical exploitation of the taxonomy emerged as the most effective strategies, with top systems consistently outperforming those that treated the two subtasks independently. CLARITY establishes political response evasion as a challenging benchmark for computational discourse analysis and highlights the difficulty of modeling strategic ambiguity in political language.

Related papers

• PolERo: Studying Political Evasion in Romanian [0.6445605125467574]
  Political evasion refers to responses that engage with a question while withholding the requested information.<n>Recent NLP work frames political evasion as a classification task using a two-level taxonomy of response clarity and fine-grained evasion strategies.<n>We introduce PolERo, a dataset of 3,574 human-annotated question-answer pairs extracted from official transcripts of five Romanian presidents.
  arXiv  Detail & Related papers  (2026-09-02T10:02:57Z)
• Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs [51.56484100374058]
  Large Language Models (LLMs) demonstrate remarkable multilingual fluency, but their internal knowledge representations remain disproportionately biased toward high-resource languages.<n>We investigate whether these biases can be mitigated at inference time, forcing an English-prompted model to answer as if it were queried in target languages.<n>Experiments on Gemma 3 12B Instruct reveal persona prompting to be the strongest overall intervention, balancing efficacy, safety, and out-of-domain generalization.
  arXiv  Detail & Related papers  (2026-07-21T16:15:05Z)
• ttda704 at SemEval-2026 Task 6: Structured Chain-of-Thought Prompting for Political Evasion Detection [0.006227637802384129]
  This paper describes our system for SemEval-2026 Task 6, which addresses the classification of political evasion strategies in English question-answer pairs extracted from U.S. presidential interviews.
  arXiv  Detail & Related papers  (2026-06-14T12:09:23Z)
• GRASP: Deterministic argument ranking in interaction graphs [65.66879897437157]
  We show that holistic judging suffers from substantial inter-model disagreement.<n>This instability arises from collapsing a debate's complex interaction structure into a single opaque score.<n>We propose GRASP, a deterministic framework that aggregates stable local interaction judgments into a global ranking.
  arXiv  Detail & Related papers  (2026-05-18T21:49:02Z)
• SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures [93.36078244548077]
  The task data consist of an extended version of our manually constructed BLEnD benchmark (Myung et al. 2024), covering more than 30 language-culture pairs.<n>As the task is designed strictly for evaluation, participants were not permitted to use the data for training, fine-tuning, few-shot learning, or any other form of model modification.<n>We report the results and present an analysis of the best-performing systems and the most commonly adopted approaches.
  arXiv  Detail & Related papers  (2026-05-04T13:49:44Z)
• Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions [0.0]
  We address Task 1 (clarity-level classification) and Task 2 (evasion-level classification)<n>Our system is based on DeBERTa-V3-base, extended with focal loss, layer-wise learning rate decay, and Duluth discourse features.<n>To address class imbalance in the training data, we augment minority classes using synthetic examples generated by Gemini 3 and Claude Sonnet 4.5.
  arXiv  Detail & Related papers  (2026-04-22T04:18:14Z)
• PolicyLLM: Towards Excellent Comprehension of Public Policy for Large Language Models [56.37206340175191]
  Large Language Models (LLMs) are increasingly integrated into real-world decision-making.<n>Yet, their ability to comprehend and reason about policy-related content remains underexplored.<n>We present textbftextitPolicyBench, the first large-scale cross-system benchmark (US-China) evaluating policy comprehension.
  arXiv  Detail & Related papers  (2026-04-14T17:27:50Z)
• More Than Sum of Its Parts: Deciphering Intent Shifts in Multimodal Hate Speech Detection [49.07221734365168]
  Hate speech on social media is critical for securing cyberspace, yet relies heavily on automated detection systems.<n>We characterize semantic intent shifts where modalities interact to construct implicit hate from benign cues or neutralize toxicity through semantic inversion.<n>We propose the Asymmetric Reasoning via Courtroom Agent DEbate framework to effectively decipher these complex cues.
  arXiv  Detail & Related papers  (2026-03-22T15:46:23Z)
• ITLC at SemEval-2026 Task 11: Normalization and Deterministic Parsing for Formal Reasoning in LLMs [9.363838558599863]
  Large language models suffer from content effects in reasoning tasks, particularly in multi-lingual contexts.<n>We introduce a novel method that reduces these biases through explicit structural abstraction.<n>Our approach achieves top-5 rankings across all subtasks while substantially reducing content effects.
  arXiv  Detail & Related papers  (2026-03-03T07:02:45Z)
• Latent Topic Synthesis: Leveraging LLMs for Electoral Ad Analysis [51.95395936342771]
  We introduce an end-to-end framework for automatically generating an interpretable topic taxonomy from an unlabeled corpus.<n>We apply this framework to a large corpus of Meta political ads from the month ahead of the 2024 U.S. Presidential election.<n>Our approach uncovers latent discourse structures, synthesizes semantically rich topic labels, and annotates topics with moral framing dimensions.
  arXiv  Detail & Related papers  (2025-10-16T20:30:20Z)
• Teaching Language Models To Gather Information Proactively [53.85419549904644]
  Large language models (LLMs) are increasingly expected to function as collaborative partners.<n>In this work, we introduce a new task paradigm: proactive information gathering.<n>We design a scalable framework that generates partially specified, real-world tasks, masking key information.<n>Within this setup, our core innovation is a reinforcement finetuning strategy that rewards questions that elicit genuinely new, implicit user information.
  arXiv  Detail & Related papers  (2025-07-28T23:50:09Z)
• "I Never Said That": A dataset, taxonomy and baselines on response clarity classification [4.16330182801919]
  We introduce a novel taxonomy that frames the task of detecting and classifying response clarity. Our proposed two-level taxonomy addresses the clarity of a response in terms of the information provided for a given question. We combine ChatGPT and human annotators to collect, validate and annotate discrete QA pairs from political interviews.
  arXiv  Detail & Related papers  (2024-09-20T20:15:06Z)
• A Provably Efficient Option-Based Algorithm for both High-Level and Low-Level Learning [54.20447310988282]
  We present a meta-algorithm alternating between regret minimization algorithms instanced at different (high and low) temporal abstractions.<n>At the higher level, we treat the problem as a Semi-Markov Decision Process (SMDP), with fixed low-level policies, while at a lower level, inner option policies are learned with a fixed high-level policy.
  arXiv  Detail & Related papers  (2024-06-21T13:17:33Z)
• Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning [56.26889258704261]
  We propose a novel "disentangled" decision-making method, Semantically Aligned task decomposition in MARL (SAMA) SAMA prompts pretrained language models with chain-of-thought that can suggest potential goals, provide suitable goal decomposition and subgoal allocation as well as self-reflection-based replanning. SAMA demonstrates considerable advantages in sample efficiency compared to state-of-the-art ASG methods.
  arXiv  Detail & Related papers  (2023-05-18T10:37:54Z)
• BEIKE NLP at SemEval-2022 Task 4: Prompt-Based Paragraph Classification for Patronizing and Condescending Language Detection [13.944149742291788]
  PCL detection task is aimed at identifying language that is patronizing or condescending towards vulnerable communities in the general media. In this paper, we give an introduction to our solution, which exploits the power of prompt-based learning on paragraph classification.
  arXiv  Detail & Related papers  (2022-08-02T08:38:47Z)
• PALI-NLP at SemEval-2022 Task 4: Discriminative Fine-tuning of Deep Transformers for Patronizing and Condescending Language Detection [4.883341580669763]
  We propose a novel Transformer-based model and its ensembles to accurately understand such language context for PCL detection. To facilitate comprehension of the subtle and subjective nature of PCL, two fine-tuning strategies are applied. The system achieves remarkable results on the official ranking, namely 1st in Subtask 1 and 5th in Subtask 2.
  arXiv  Detail & Related papers  (2022-03-09T10:05:10Z)
• Learning Invariable Semantical Representation from Language for Extensible Policy Generalization [4.457682773596843]
  We propose a method to learn semantically invariant representations called element randomization. We theoretically prove the feasibility of learning semantically invariant representations through randomization. Experiments on challenging long-horizon tasks show that our low-level policy reliably generalizes to tasks against environment changes.
  arXiv  Detail & Related papers  (2022-01-26T08:04:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.