Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection

• URL: http://arxiv.org/abs/2607.17382v1
• Date: Sun, 19 Jul 2026 18:48:59 GMT
• Title: Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection
• Abstract summary: Existing research shows that AI-generated text detection classifiers achieve strong in-distribution (ID) performance but do not maintain the same performance on out-of-distribution (OOD) texts.<n>We fine-tune BERT-tiny models with Bayesian classification heads to select texts across three different datasets to use as a consolidated training set.<n>Our results highlight that careful dataset curation can lead to strong OOD performance.
• Score: 0.33842793760651557
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing research shows that AI-generated text detection classifiers achieve strong in-distribution (ID) performance but do not maintain the same performance on out-of-distribution (OOD) texts, suggesting overfitting to dataset-specific features. However, combining different training datasets doesn't always improve performance and, in some cases, can even encourage shortcut learning. To address this issue, we fine-tune BERT-tiny models with Bayesian classification heads to select texts across three different datasets to use as a consolidated training set. We trained three different classifiers: fine-tuned DeBERTa-V3-large and ModernBERT-large classifiers via empirical X-risk minimization, and an MCGrad model that calibrates the predictions from the ModernBERT-large classifier. The DeBERTa-V3-large-large classifier achieves a mean score of 0.882 on the PAN 2026 test set across five metrics: AUROC, $F_1$, C@1, Brier score, and $F_{0.5u}$. ModernBERT-large achieves a score of 0.96 while MCGrad achieves the best score of the three with a mean score of 0.974, ranking second on the leaderboard. Our results highlight that careful dataset curation can lead to strong OOD performance. We release our ModernBERT-large and DeBERTa-V3-large models at https://huggingface.co/collections/ShantanuT01/panclef-2026 .

Related papers

• Arena-T2I Hard: Benchmarking and Improving Faithfulness with Dependency-Aware Checklist [69.81453995715673]
  Existing faithfulness benchmarks rely on simple atomic instructions.<n>We introduce Arena-T2I Hard, a 310-prompt stress benchmark drawn from real arena T2I logs.<n>We propose a dependency-aware checklist reward that decomposes each prompt into a DAG of yes/no questions and zeroes descendants of failed parents.
  arXiv  Detail & Related papers  (2026-06-30T14:17:05Z)
• Frozen Multimodal Embeddings for AI-Assisted Interview Assessment of Personality and Cognitive Ability [0.20999222360659606]
  This paper presents our solution for the ACM Multimedia AVI Challenge 2026.<n>Track1 predicts self-reported HEXACO personality traits from personality-related interview responses, and Track2 classifies cognitive ability levels.<n>We use frozen multimodal encoders, including CLIP for visual features, Whisper for acoustic features and transcripts, and RoBERTa, E5, and DeBERTaV3 for textual representations.
  arXiv  Detail & Related papers  (2026-06-10T11:03:34Z)
• TriTopic: Tri-Modal Graph-Based Topic Modeling with Iterative Refinement and Archetypes [0.40611352512781873]
  We present TriTopic, a framework that addresses these weaknesses through a tri-modal graph fusing semantic embeddings, TF-IDF, and metadata.<n>TriTopic replaces the "average document" concept with archetype-based topic representations defined by boundary cases rather than centers alone.<n>In benchmarks across 20 Newsgroups, BBC News, AG News, and Arxiv, TriTopic achieves the highest NMI on every dataset.
  arXiv  Detail & Related papers  (2026-02-22T07:29:53Z)
• Back to Supervision: Boosting Word Boundary Detection through Frame Classification [5.901031474087739]
  We propose a model-agnostic framework to perform word boundary detection in a supervised manner. We trained and tested on the Buckeye dataset and only tested on TIMIT one. We achieved F-values of 0.8427 on the Buckeye dataset and 0.7436 on the TIMIT dataset, along with R-values of 0.8489 and 0.7807, respectively.
  arXiv  Detail & Related papers  (2024-11-15T18:43:29Z)
• Performance Evaluation of Deep Learning and Transformer Models Using Multimodal Data for Breast Cancer Classification [7.405837346783951]
  Rising breast cancer (BC) occurrence and mortality are major global concerns for women. Deep learning (DL) has demonstrated superior diagnostic performance in BC classification compared to human expert readers. This study proposes a multimodal DL architecture for BC classification, utilising images (mammograms; four views) and textual data (radiological reports) from our new in-house dataset.
  arXiv  Detail & Related papers  (2024-10-14T04:22:24Z)
• Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values [73.82043713141142]
  Many NLP classification tasks, such as sexism/racism detection or toxicity detection, are based on human values. We introduce a framework for value-aligned classification that performs prediction based on explicitly written human values in the command.
  arXiv  Detail & Related papers  (2022-10-14T09:10:49Z)
• Efficient, Uncertainty-based Moderation of Neural Networks Text Classifiers [8.883733362171034]
  We propose a framework for the efficient, in-operation moderation of classifiers' output. We suggest a semi-automated approach that uses prediction uncertainties to pass unconfident, probably incorrect classifications to human moderators. A series of benchmarking experiments show that our framework can improve the classification F1-scores by 5.1 to 11.2%.
  arXiv  Detail & Related papers  (2022-04-04T09:07:54Z)
• MSeg: A Composite Dataset for Multi-domain Semantic Segmentation [100.17755160696939]
  We present MSeg, a composite dataset that unifies semantic segmentation datasets from different domains. We reconcile the generalization and bring the pixel-level annotations into alignment by relabeling more than 220,000 object masks in more than 80,000 images. A model trained on MSeg ranks first on the WildDash-v1 leaderboard for robust semantic segmentation, with no exposure to WildDash data during training.
  arXiv  Detail & Related papers  (2021-12-27T16:16:35Z)
• DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing [117.41016786835452]
  This paper presents a new pre-trained language model, DeBERTaV3, which improves the original DeBERTa model. vanilla embedding sharing in ELECTRA hurts training efficiency and model performance. We propose a new gradient-disentangled embedding sharing method that avoids the tug-of-war dynamics.
  arXiv  Detail & Related papers  (2021-11-18T06:48:00Z)
• HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units [81.53783563025084]
  We propose an offline clustering step to provide aligned target labels for a BERT-like prediction loss. A key ingredient of our approach is applying the prediction loss over the masked regions only. HuBERT shows up to 19% and 13% relative WER reduction on the more challenging dev-other and test-other evaluation subsets.
  arXiv  Detail & Related papers  (2021-06-14T14:14:28Z)
• NLRG at SemEval-2021 Task 5: Toxic Spans Detection Leveraging BERT-based Token Classification and Span Prediction Techniques [0.6850683267295249]
  In this paper, we explore simple versions of Token Classification or Span Prediction approaches. We use BERT-based models -- BERT, RoBERTa, and SpanBERT for both approaches. To this end, we investigate results on four hybrid approaches -- Multi-Span, Span+Token, LSTM-CRF, and a combination of predicted offsets using union/intersection.
  arXiv  Detail & Related papers  (2021-02-24T12:30:09Z)
• TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task [80.38130122127882]
  TACRED is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE) In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement? We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled.
  arXiv  Detail & Related papers  (2020-04-30T15:07:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.