LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification

• URL: http://arxiv.org/abs/2606.00647v1
• Date: Sat, 30 May 2026 09:47:29 GMT
• Title: LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification
• Abstract summary: For the PsyDefDetect 2026 shared task, our team LinguIUTics achieves a macro F1-score of 0.3917 on the official positive-class leaderboard.<n>We leverage three key strategies: grouped stratified cross-validation, minority-class round-robin lexical augmentation, and a post-processing pipeline with logit bias tuning and ensemble blending.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Detecting psychological defense mechanisms in conversational text remains a challenging clinical NLP problem. For the PsyDefDetect 2026 shared task (nine-class utterance classification evaluated via macro F1), our team LinguIUTics achieves a macro F1-score of 0.3917 on the official positive-class leaderboard, ranking 4th out of 21 registered teams and improving over the Ministral-8B task baseline (31.48 macro F1) by 7.7 absolute points (24.4 percent relative). BERT-family encoders and zero-shot LLMs proved ineffective on rare classes due to severe class imbalance, leading us to QLoRA fine-tuning of Qwen3-8B. We leverage three key strategies: grouped stratified cross-validation (preventing leakage), minority-class round-robin lexical augmentation, and a post-processing pipeline with logit bias tuning and ensemble blending. Together, these components close much of the validation-to-leaderboard gap and substantially improve minority-class recall, driving the critical "Unclear" class (Level 8) from near-zero performance to an F1 score of 0.797.

Related papers

• Cybersecurity Detection Classification with Reasoning-enabled Language Models [50.72675435043546]
  A major issue in Security Operations Centers (SOCs) is alert fatigue.<n>Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat.<n>We train a chain-of-thought (CoT) reasoning-enabled triage classifier on real, human-labeled Windows endpoint detections.
  arXiv  Detail & Related papers  (2026-07-30T16:22:13Z)
• Separating Secrets from Placeholders: A Hybrid CNN-CodeBERT Framework for Three-Class Credential Leakage Detection [3.853925623717688]
  Credential leakage in public source code repositories poses a critical security threat.<n>We propose a three-class classification framework that explicitly models placeholder or weak credentials as a distinct class.<n>We evaluate our approach on a newly constructed dataset of 9,426 samples spanning 10 programming languages.
  arXiv  Detail & Related papers  (2026-05-29T16:36:20Z)
• Overview of the PsyDefDetect Shared Task at BioNLP 2026: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations [33.49851288070166]
  We present an overview of PsyDefDetect, the shared task on detecting levels of psychological defense mechanisms in emotional support dialogues.<n>The task asks systems to classify a target seeker utterance into one of nine categories: seven hierarchical Defense Mechanism Rating Scales (DMRS) levels plus two auxiliary labels.<n>The best system achieved a macro F1-score of 0.420, surpassing the strongest fine-tuned baseline reported in the dataset paper by a notable margin.
  arXiv  Detail & Related papers  (2026-05-24T07:17:54Z)
• Step-wise Rubric Rewards for LLM Reasoning [72.17879367869503]
  Reinforcement Learning with Verifiable Rewards (RLVR) is widely used to improve reasoning in large language models.<n>We find 18.2% of steps in correct responses are wrong yet positively rewarded.<n>Across six mathematical reasoning benchmarks, SRaR improves average accuracy over RaR by 3.57 points.
  arXiv  Detail & Related papers  (2026-05-17T07:08:14Z)
• UTS at PsyDefDetect: Multi-Agent Councils and Absence-Based Reasoning for Defense Mechanism Classification [1.348563827917271]
  This paper describes our system for classifying psychological defense mechanisms in emotional support dialogues.<n>A central insight is that defense mechanisms are defined by what is absent: missing affect, blocked cognition, denied reality.<n>We encode this as an affect-cognition integration spectrum in prompt-level clinical rules, which account for the largest single gain.
  arXiv  Detail & Related papers  (2026-05-10T21:30:02Z)
• Nürnberg NLP at PsyDefDetect: Multi-Axis Voter Ensembles for Psychological Defence Mechanism Classification [0.0]
  In PsyDefDetect the eight positive defence categories share surface language and differ only in pragmatic function.<n>We translate this insight into a 9-voter ensemble spanning three axes.<n>The system reaches $Ftest=.420$ on the hidden test set, placing first among 21 registered teams.
  arXiv  Detail & Related papers  (2026-05-08T11:32:31Z)
• UCSC-NLP at SemEval-2026 Task 13: Multi-View Generalization and Diagnostic Analysis of Machine-Generated Code Detection [0.0]
  We present our system for distinguishing between human-written and AI-generated code.<n>For Subtask A, we fine-tune UniXcoder-base with a multi-view training framework that promotes generator-invariant representations.<n>For Subtask B, we show that severe class imbalance causes catastrophic minority-class failure under standard fine-tuning.
  arXiv  Detail & Related papers  (2026-04-28T21:41:59Z)
• Do We Still Need Humans in the Loop? Comparing Human and LLM Annotation in Active Learning for Hostility Detection [68.37351671559675]
  Active learning can annotate thousands of instances from a short prompt at negligible cost.<n>This raises two questions for active learning (AL): can LLM labels replace human labels within the AL loop, and does AL remain necessary when entire corpora can be labelled at once?<n>We investigate both questions on a new dataset of 277,902 German political TikTok comments.
  arXiv  Detail & Related papers  (2026-04-15T14:10:58Z)
• Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback [59.078756231841574]
  Critique-GRPO is an online RL framework that integrates both natural language and numerical feedback for effective policy optimization.<n>We show Critique-GRPO consistently outperforms supervised learning and RL-based fine-tuning methods across eight challenging mathematical, STEM, and general reasoning tasks.
  arXiv  Detail & Related papers  (2025-06-03T17:39:02Z)
• Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness [61.45587642780908]
  We propose a three-step approach for parameter-efficient fine-tuning of image-text foundation models.<n>Our method improves its two key components: minority samples identification and the robust training algorithm.<n>Our theoretical analysis shows that our PPA enhances minority group identification and is Bayes optimal for minimizing the balanced group error.
  arXiv  Detail & Related papers  (2025-03-12T15:46:12Z)
• Revisiting adversarial training for the worst-performing class [60.231877895663956]
  There is a substantial gap between the top-performing and worst-performing classes in many datasets. We argue that this gap can be reduced by explicitly optimizing for the worst-performing class. Our method, called class focused online learning (CFOL), includes high probability convergence guarantees for the worst class loss.
  arXiv  Detail & Related papers  (2023-02-17T13:41:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.