DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models

• URL: http://arxiv.org/abs/2607.26891v1
• Date: Wed, 29 Jul 2026 13:23:30 GMT
• Title: DIRECT: Direct Decoding for Efficient and Aligned Sequence Labeling with Large Language Models
• Abstract summary: Sequence labeling is a fine-grained information extraction task.<n>We propose DIRECT, a framework that addresses these issues through training-time optimization and inference-time rectification.<n> DIRECT achieves significant improvements in both performance and efficiency compared to existing methods.
• Score: 15.41671030303189
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Sequence labeling is a fine-grained information extraction task, yet existing large language model-based approaches suffer from insufficient domain alignment and low inference efficiency. To address these issues, we propose DIRECT, a framework that addresses these issues through training-time optimization and inference-time rectification. Specifically, DIRECT performs Direct Preference Optimization (DPO) after supervised fine-tuning to strengthen task alignment with human preferences, and introduces a controlled decoding process that enforces fixed output formats and restricts predictions to candidate sets. To further improve efficiency, a template-filling mechanism requires the model to generate only label tokens while reusing prefixed content through the KV Cache, thus reducing redundant computation. Experimental results on eight datasets demonstrate that DIRECT achieves significant improvements in both performance and efficiency compared to existing methods.

Related papers

• CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization [0.0]
  Large Language Models utilizing reasoning techniques improve task performance but incur significant latency and token costs due to verbose generation.<n>We propose Cost-Regularized Optimization of Prompts (CROP), an APO method that introduces regularization on response length.<n>We evaluate our approach on complex reasoning datasets, specifically GSM8K, LogiQA and BIG-Bench Hard.
  arXiv  Detail & Related papers  (2026-04-08T19:31:25Z)
• Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models [102.20309135516186]
  Cross-entropy (CE) training provides dense and scalable supervision for language models.<n>We introduce a feature-matching objective for language-model fine-tuning.<n>To optimize this objective efficiently, we propose energy-based fine-tuning.
  arXiv  Detail & Related papers  (2026-03-12T17:57:50Z)
• Semantic Prioritization in Visual Counterfactual Explanations with Weighted Segmentation and Auto-Adaptive Region Selection [50.68751788132789]
  This study introduces an innovative methodology named as Weighted Semantic Map with Auto-adaptive Candidate Editing Network (WSAE-Net)<n>The generation of the weighted semantic map is designed to maximize the reduction of non-semantic feature units that need to be computed.<n>The auto-adaptive candidate editing sequences are designed to determine the optimal computational order among the feature units to be processed.
  arXiv  Detail & Related papers  (2025-11-17T05:34:10Z)
• PrefixAgent: An LLM-Powered Design Framework for Efficient Prefix Adder Optimization [11.652417515791633]
  PrefixAgent is a large language model (LLM)-powered framework for efficient prefix adder optimization.<n>It reformulates the problem into subtasks including backbone synthesis and structure refinement, which effectively reduces the search space.<n> Experimental results show that PrefixAgent synthesizes prefix adders with consistently smaller areas compared to baseline methods.
  arXiv  Detail & Related papers  (2025-07-08T16:14:17Z)
• Evolutionary Guided Decoding: Iterative Value Refinement for LLMs [41.56764640311065]
  Iterative Value Refinement is a novel framework designed to bridge this gap.<n>It employs Value Exploration to provide a more comprehensive and robust training signal.<n>Iterative Self-Refinement uses the improved value function from one iteration to guide the generation of higher-quality data.
  arXiv  Detail & Related papers  (2025-03-04T07:49:10Z)
• PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selection [68.8373788348678]
  Visual instruction tuning adapts pre-trained Multimodal Large Language Models to follow human instructions.<n>PRISM is the first training-free framework for efficient visual instruction selection.<n>It reduces the end-to-end time for data selection and model tuning to just 30% of conventional pipelines.
  arXiv  Detail & Related papers  (2025-02-17T18:43:41Z)
• DRPruning: Efficient Large Language Model Pruning through Distributionally Robust Optimization [59.96455188197593]
  Large language models (LLMs) deliver impressive results but face challenges from increasing model sizes and computational costs.<n>We propose DRPruning, a method that dynamically adjusts the data distribution during training to restore balanced performance across heterogeneous and multi-tasking data.<n> Experiments in monolingual and multilingual settings show that DRPruning surpasses similarly sized models in both pruning and continued pretraining over perplexity, downstream tasks, and instruction tuning.
  arXiv  Detail & Related papers  (2024-11-21T12:02:39Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• Adaptive Draft-Verification for Efficient Large Language Model Decoding [24.347886232342862]
  Large language model (LLM) decoding involves generating a sequence of tokens based on a given context. The typical autoregressive decoding method requires a separate forward pass through the model for each token generated. We introduce ADED, which accelerates LLM decoding without requiring fine-tuning.
  arXiv  Detail & Related papers  (2024-06-27T22:20:39Z)
• PREM: A Simple Yet Effective Approach for Node-Level Graph Anomaly Detection [65.24854366973794]
  Node-level graph anomaly detection (GAD) plays a critical role in identifying anomalous nodes from graph-structured data in domains such as medicine, social networks, and e-commerce. We introduce a simple method termed PREprocessing and Matching (PREM for short) to improve the efficiency of GAD. Our approach streamlines GAD, reducing time and memory consumption while maintaining powerful anomaly detection capabilities.
  arXiv  Detail & Related papers  (2023-10-18T02:59:57Z)
• Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization [94.4409074435894]
  We propose a novel and effective fine-tuning framework, named Layerwise Noise Stability Regularization (LNSR) Specifically, we propose to inject the standard Gaussian noise and regularize hidden representations of the fine-tuned model. We demonstrate the advantages of the proposed method over other state-of-the-art algorithms including L2-SP, Mixout and SMART.
  arXiv  Detail & Related papers  (2022-06-12T04:42:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.