Divide, Cache, Conquer: Dichotomic Prompting for Efficient Multi-Label LLM-Based Classification

• URL: http://arxiv.org/abs/2511.03830v1
• Date: Wed, 05 Nov 2025 19:53:51 GMT
• Title: Divide, Cache, Conquer: Dichotomic Prompting for Efficient Multi-Label LLM-Based Classification
• Authors: Mikołaj Langner, Jan Eliasz, Ewa Rudnicka, Jan Kocoń,
• Abstract summary: We introduce a method for efficient multi-label text classification with large language models (LLMs)<n>Instead of generating all labels in a single structured response, each target dimension is queried independently.<n>Our findings suggest that decomposing multi-label classification into dichotomic queries offers a scalable and effective framework.
• Score: 0.2799896314754614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a method for efficient multi-label text classification with large language models (LLMs), built on reformulating classification tasks as sequences of dichotomic (yes/no) decisions. Instead of generating all labels in a single structured response, each target dimension is queried independently, which, combined with a prefix caching mechanism, yields substantial efficiency gains for short-text inference without loss of accuracy. To demonstrate the approach, we focus on affective text analysis, covering 24 dimensions including emotions and sentiment. Using LLM-to-SLM distillation, a powerful annotator model (DeepSeek-V3) provides multiple annotations per text, which are aggregated to fine-tune smaller models (HerBERT-Large, CLARIN-1B, PLLuM-8B, Gemma3-1B). The fine-tuned models show significant improvements over zero-shot baselines, particularly on the dimensions seen during training. Our findings suggest that decomposing multi-label classification into dichotomic queries, combined with distillation and cache-aware inference, offers a scalable and effective framework for LLM-based classification. While we validate the method on affective states, the approach is general and applicable across domains.

Related papers

• Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Counting [59.37613121962146]
  We propose WS-COC, the first MLLM-driven weakly-supervised framework for class-agnostic object counting.<n> WS-COC matches or even surpasses many state-of-art fully-supervised methods while significantly reducing annotation costs.
  arXiv  Detail & Related papers  (2026-02-13T09:58:35Z)
• Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches [0.0]
  We explore strategies to fine-tune decoder-only Large Language Models (LLMs) for downstream text classification under resource constraints.<n>Two approaches are investigated: (1) attaching a classification head to a pre-trained causal LLM and fine-tuning on the task, and (2) instruction-tuning the LLM in a prompt->response format for classification.
  arXiv  Detail & Related papers  (2025-12-14T13:02:06Z)
• RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns [50.401907401444404]
  Large language models (LLMs) are crucial for preventing misuse and building trustworthy AI systems.<n>We propose RepreGuard, an efficient statistics-based detection method.<n> Experimental results show that RepreGuard outperforms all baselines with average 94.92% AUROC on both in-distribution (ID) and OOD scenarios.
  arXiv  Detail & Related papers  (2025-08-18T17:59:15Z)
• An Enhanced Model-based Approach for Short Text Clustering [58.60681789677676]
  Short text clustering has become increasingly important with the popularity of social media like Twitter, Google+, and Facebook.<n>Existing methods can be broadly categorized into two paradigms: topic model-based approaches and deep representation learning-based approaches.<n>We propose a collapsed Gibbs Sampling algorithm for the Dirichlet Multinomial Mixture model (GSDMM), which effectively handles the sparsity and high dimensionality of short texts.<n>Based on several aspects of GSDMM that warrant further refinement, we propose an improved approach, GSDMM+, designed to further optimize its performance.
  arXiv  Detail & Related papers  (2025-07-18T10:07:42Z)
• Label-semantics Aware Generative Approach for Domain-Agnostic Multilabel Classification [25.4439290862464]
  We introduce a robust, efficient domain-agnostic generative model framework for multi-label text classification.<n>Our approach utilizes predefined label descriptions and is trained to generate these descriptions based on the input text.<n>We demonstrate the effectiveness of our proposed model by achieving new state-of-the-art performances across all evaluated datasets.
  arXiv  Detail & Related papers  (2025-06-07T14:07:07Z)
• Large Language Models Do Multi-Label Classification Differently [41.60681320369492]
  Multi-label classification is prevalent in real-world settings, but the behavior of Large Language Models (LLMs) in this setting is understudied.<n>We investigate how autoregressive LLMs perform multi-label classification, focusing on subjective tasks.<n>We find that the initial probability distribution for the first label often does not reflect the eventual final output.
  arXiv  Detail & Related papers  (2025-05-23T06:04:36Z)
• Label-template based Few-Shot Text Classification with Contrastive Learning [7.964862748983985]
  We propose a simple and effective few-shot text classification framework.<n>Label templates are embedded into input sentences to fully utilize the potential value of class labels.<n> supervised contrastive learning is utilized to model the interaction information between support samples and query samples.
  arXiv  Detail & Related papers  (2024-12-13T12:51:50Z)
• Boosting Prompt-Based Self-Training With Mapping-Free Automatic Verbalizer for Multi-Class Classification [3.647905567437244]
  We introduce a novel, efficient verbalizer structure, named Mapping-free Automatic Verbal Modelingizer (MAV) MAV serves as a trainable verbalizer that automatically extracts the requisite word features for classification by capitalizing on all available information from predictions. Experimental results on five multi-class classification datasets indicate MAV's superior self-training efficacy.
  arXiv  Detail & Related papers  (2023-12-08T11:43:00Z)
• SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification [6.227343685358882]
  We present a model-agnostic framework that sparsifies and integrates internal neurons of intermediate layers of Large Language Models for text classification. SPIN significantly improves text classification accuracy, efficiency, and interpretability.
  arXiv  Detail & Related papers  (2023-11-27T16:28:20Z)
• Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignment [53.2701026843921]
  Large-scale pre-trained Vision Language Models (VLMs) have proven effective for zero-shot classification. In this paper, we aim at a more challenging setting, Realistic Zero-Shot Classification, which assumes no annotation but instead a broad vocabulary. We propose the Self Structural Semantic Alignment (S3A) framework, which extracts structural semantic information from unlabeled data while simultaneously self-learning.
  arXiv  Detail & Related papers  (2023-08-24T17:56:46Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• Revisiting Self-Training for Few-Shot Learning of Language Model [61.173976954360334]
  Unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model. In this work, we revisit the self-training technique for language model fine-tuning and present a state-of-the-art prompt-based few-shot learner, SFLM.
  arXiv  Detail & Related papers  (2021-10-04T08:51:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.