LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models

• URL: http://arxiv.org/abs/2603.19255v1
• Date: Wed, 25 Feb 2026 15:34:21 GMT
• Title: LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models
• Abstract summary: We propose LARFT (Length-Aware Reinforcement Fine-Tuning), a training framework that aligns the model's length cognition with its action.<n>LARFT integrates length-oriented reinforcement learning with a hindsight length awareness.<n>Experiments demonstrate that LARFT outperforms existing baselines, achieving an average improvement of +20.92 points across three length instruction following benchmarks.
• Score: 13.817055649196107
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the strong performance of Large Language Models (LLMs) on complex instruction-following tasks, precise control of output length remains a persistent challenge. Existing methods primarily attempt to enforce length constraints by externally imposing length signals or optimization objectives, while largely overlooking the underlying limitation: the model's intrinsic deficit in length cognition. To address this, we propose LARFT (Length-Aware Reinforcement Fine-Tuning), a training framework that aligns the model's length cognition with its action. Specifically, LARFT integrates length-oriented reinforcement learning with a hindsight length awareness. By transforming on-policy data into hindsight self-awareness tasks where the model learns to identify the actual length of its own generation, LARFT jointly optimizes the model's internal representation of length information and refines its policy to satisfy length constraints, thereby achieving precise and reliable length instruction following. Extensive experiments across four base models demonstrate that LARFT outperforms existing baselines, achieving an average improvement of +20.92 points across three length instruction following benchmarks with only a marginal decline of -1.45 points on four general capability benchmarks.

Related papers

• OPSDL: On-Policy Self-Distillation for Long-Context Language Models [3.2617036218058413]
  OPSDL is an On-Policy Self-Distillation method for enhancing the Long-context capabilities of large language models.<n>We evaluate OPSDL on long-context benchmarks across a range of models from 7B to 32B parameters.
  arXiv  Detail & Related papers  (2026-04-19T16:53:56Z)
• Length-Adaptive Interest Network for Balancing Long and Short Sequence Modeling in CTR Prediction [50.094751096858204]
  LAIN is a plug-and-play framework that incorporates sequence length as a conditioning signal to balance long- and short-sequence modeling.<n>Our work offers a general, efficient, and deployable solution to mitigate length-induced bias in sequential recommendation.
  arXiv  Detail & Related papers  (2026-01-27T03:14:20Z)
• Plan-and-Write: Structure-Guided Length Control for LLMs without Model Retraining [2.168162018395079]
  This paper presents a prompt engineering methodology that enables precise length control without model retraining.<n>Our structure-guided approach implements deliberate planning and word counting mechanisms within the prompt, encouraging the model to carefully track and adhere to specified length constraints.<n>Our approach provides an immediately deployable solution for applications requiring precise length control, particularly valuable for production environments where model retraining is impractical or cost-prohibitive.
  arXiv  Detail & Related papers  (2025-11-03T18:10:42Z)
• Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models [74.15250326312179]
  Diffusion Large Language Models offer efficient parallel generation and capable global modeling.<n>The dominant application ofDLLMs is hindered by the need for a statically predefined generation length.<n>We introduce DAEDAL, a novel training-free denoising strategy that enables Dynamic Adaptive Length Expansion.
  arXiv  Detail & Related papers  (2025-08-01T17:56:07Z)
• Long-Short Alignment for Effective Long-Context Modeling in LLMs [32.13785291956956]
  Large language models (LLMs) have exhibited impressive performance and surprising emergent properties.<n>Length generalization -- the ability to generalize to sequences longer than those seen during training -- is a classical and fundamental problem.<n>We highlight the critical role of textbflong-short alignment -- the consistency of output distributions across sequences of varying lengths.
  arXiv  Detail & Related papers  (2025-06-13T13:25:39Z)
• UNCLE: Benchmarking Uncertainty Expressions in Long-Form Generation [69.13972417517233]
  Large Language Models (LLMs) are prone to hallucination, particularly in long-form generations.<n>We introduce UNCLE, a benchmark designed to evaluate uncertainty expression in both long- and short-form question answering (QA)<n>Our dataset is the first to directly link short- and long-form QA through aligned questions and gold-standard answers.
  arXiv  Detail & Related papers  (2025-05-22T17:16:08Z)
• Disentangling Length Bias In Preference Learning Via Response-Conditioned Modeling [87.17041933863041]
  Reinforcement Learning from Human Feedback (RLHF) has achieved considerable success in aligning large language models (LLMs)<n>We introduce a $textbfR$esponse-$textbfc$onditioned $textbfB$radley-$textbfT$erry (Rc-BT) model that enhances the model's capability in length bias mitigating and length instruction following.<n>We also propose the Rc-RM and Rc-DPO algorithm to leverage the Rc-BT model for reward modeling and direct policy optimization
  arXiv  Detail & Related papers  (2025-02-02T14:50:25Z)
• Ruler: A Model-Agnostic Method to Control Generated Length for Large Language Models [14.175953642749649]
  Large language models often struggle to generate responses of a specific length. We introduce a novel, model-agnostic approach called Ruler to enhance the instruction-following ability of large language models under length-constrained instructions.
  arXiv  Detail & Related papers  (2024-09-27T17:44:58Z)
• Long Context Alignment with Short Instructions and Synthesized Positions [56.1267385315404]
  This paper introduces Step-Skipping Alignment (SkipAlign) It is a new technique designed to enhance the long-context capabilities of Large Language Models (LLMs) With a careful selection of the base model and alignment datasets, SkipAlign with only 6B parameters achieves it's best performance and comparable with strong baselines like GPT-3.5-Turbo-16K on LongBench.
  arXiv  Detail & Related papers  (2024-05-07T01:56:22Z)
• CLEX: Continuous Length Extrapolation for Large Language Models [68.43814043853347]
  We propose Continuous Length EXtrapolation (CLEX) for Large Language Models (LLMs) CLEX extends the context window to over 4x or almost 8x training length, with no deterioration in performance. Our model trained on a 4k length exhibits competitive performance against state-of-the-art open-source models trained on context lengths up to 32k.
  arXiv  Detail & Related papers  (2023-10-25T08:13:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.