Discrete Diffusion Language Models Are Training-Free Multi-Label Classifiers

• URL: http://arxiv.org/abs/2608.14649v1
• Date: Thu, 30 Jul 2026 15:21:02 GMT
• Title: Discrete Diffusion Language Models Are Training-Free Multi-Label Classifiers
• Abstract summary: We present dLLM-SetScore, a training-free method that uses discrete masked-diffusion language models for multi-label text classification.<n>For each candidate label, it asks a short yes/no question and compares the probabilities of the two answer tokens at one masked position.
• Score: 1.3406606730449597
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present dLLM-SetScore, a training-free method that uses discrete masked-diffusion language models for multi-label text classification. For each candidate label, it asks a short yes/no question and compares the probabilities of the two answer tokens at one masked position. The method uses no task-specific fine-tuning or training on textual-entailment datasets; a 200-example labelled validation slice selects thresholds, temperature, and prompt wording. We first show that placing all labels in one prompt creates a strong slot-position asymmetry: the first answer slot is predicted positive on $99.4\%$ of GoEmotions examples and $100\%$ of Reuters examples. Per-label scoring places every label in the same syntactic position, making predictions invariant to label order and avoiding this artifact. We evaluate LLaDA-8B and Dream-7B on six datasets against NLI models, an autoregressive LLM, SetFit, and supervised classifiers. On the five datasets shared by both diffusion families, Instruct checkpoints improve macro-F1 in 9 of 10 comparisons and micro-F1 in 8 of 10, although these comparisons do not identify the cause. Within our protocol, LLaDA-Instruct records the highest training-free values for both Reuters and ECtHR metrics. We prove permutation invariance, characterize thresholded decisions under weighted Hamming loss, and derive shortlist ceilings for recall and F1. An exploratory local Joint Set Refinement step lowers F1 from biased and unbiased initializations and is retained as a negative result.

Related papers

• Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models [59.51249894128724]
  Masked diffusion language models can reduce inference steps by revealing multiple tokens per denoising iteration.<n>We propose ADAS, a training-free reranking rule for parallel masked diffusion decoding.
  arXiv  Detail & Related papers  (2026-06-09T13:17:27Z)
• RAPT: Retrieval-Augmented Post-hoc Thresholding for Multi-Label Classification [1.2599533416395765]
  We present RAPT, a deployment-oriented retrieval-augmented score thresholding wrapper.<n>RAPT is a model-agnostic wrapper for document representations for similarity search and per label confidence scores.<n>In industrial setting, RAPT achieved its best predictive performance with metric learners, reaching 0.87 Macro-F1.
  arXiv  Detail & Related papers  (2026-05-15T18:31:04Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios [58.617025733655005]
  We propose a vision-language prompt tuning method with mitigated label bias (M-Tuning)<n>It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more, and thus prompts are tuned in a simulated open-set scenario.<n>Our method achieves the best performance on datasets with various scales, and extensive ablation studies also validate its effectiveness.
  arXiv  Detail & Related papers  (2023-03-09T09:05:47Z)
• Multi-label Classification with Partial Annotations using Class-aware Selective Loss [14.3159150577502]
  Large-scale multi-label classification datasets are commonly partially annotated. We analyze the partial labeling problem, then propose a solution based on two key ideas. With our novel approach, we achieve state-of-the-art results on OpenImages dataset.
  arXiv  Detail & Related papers  (2021-10-21T08:10:55Z)
• Detecting Handwritten Mathematical Terms with Sensor Based Data [71.84852429039881]
  We propose a solution to the UbiComp 2021 Challenge by Stabilo in which handwritten mathematical terms are supposed to be automatically classified. The input data set contains data of different writers, with label strings constructed from a total of 15 different possible characters.
  arXiv  Detail & Related papers  (2021-09-12T19:33:34Z)
• PLM: Partial Label Masking for Imbalanced Multi-label Classification [59.68444804243782]
  Neural networks trained on real-world datasets with long-tailed label distributions are biased towards frequent classes and perform poorly on infrequent classes. We propose a method, Partial Label Masking (PLM), which utilizes this ratio during training. Our method achieves strong performance when compared to existing methods on both multi-label (MultiMNIST and MSCOCO) and single-label (imbalanced CIFAR-10 and CIFAR-100) image classification datasets.
  arXiv  Detail & Related papers  (2021-05-22T18:07:56Z)
• Coping with Label Shift via Distributionally Robust Optimisation [72.80971421083937]
  We propose a model that minimises an objective based on distributionally robust optimisation (DRO) We then design and analyse a gradient descent-proximal mirror ascent algorithm tailored for large-scale problems to optimise the proposed objective.
  arXiv  Detail & Related papers  (2020-10-23T08:33:04Z)
• Instance Credibility Inference for Few-Shot Learning [45.577880041135785]
  Few-shot learning aims to recognize new objects with extremely limited training data for each category. This paper presents a simple statistical approach, dubbed Instance Credibility Inference (ICI) to exploit the distribution support of unlabeled instances for few-shot learning. Our simple approach can establish new state-of-the-arts on four widely used few-shot learning benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-26T12:01:15Z)
• Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation [35.593312267921256]
  Like humans, deep networks have been shown to learn better when samples are organized and introduced in a meaningful order or curriculum. We propose Learning with Incremental Labels and Adaptive Compensation (LILAC), a two-phase method that incrementally increases the number of unique output labels.
  arXiv  Detail & Related papers  (2020-01-13T21:00:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.