Hansel: Output Length Controlling Framework for Large Language Models

• URL: http://arxiv.org/abs/2412.14033v1
• Date: Wed, 18 Dec 2024 16:52:38 GMT
• Title: Hansel: Output Length Controlling Framework for Large Language Models
• Abstract summary: Hansel is an efficient framework for length control in large language models.<n>The framework can be applied to any pre-trained LLMs during the finetuning stage of the model.
• Score: 3.053720182019654
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the great success of large language models (LLMs), efficiently controlling the length of the output sequence still remains a challenge. In this paper, we propose Hansel, an efficient framework for length control in LLMs without affecting its generation ability. Hansel utilizes periodically outputted hidden special tokens to keep track of the remaining target length of the output sequence. Together with techniques to avoid abrupt termination of the output, this seemingly simple method proved to be efficient and versatile, while not harming the coherency and fluency of the generated text. The framework can be applied to any pre-trained LLMs during the finetuning stage of the model, regardless of its original positional encoding method. We demonstrate this by finetuning four different LLMs with Hansel and show that the mean absolute error of the output sequence decreases significantly in every model and dataset compared to the prompt-based length control finetuning. Moreover, the framework showed a substantially improved ability to extrapolate to target lengths unseen during finetuning, such as long dialog responses or extremely short summaries. This indicates that the model learns the general means of length control, rather than learning to match output lengths to those seen during training.

Related papers

• Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulation [50.821215666749545]
  Large Language Models (LLMs) often fail to accurately measure their response lengths, leading to poor adherence to length constraints.<n>We propose a novel length regulation approach that incorporates dynamic length feedback during generation.<n> Experiments on summarization and biography tasks show our training-free approach significantly improves precision in achieving target token, word, or sentence counts.
  arXiv  Detail & Related papers  (2026-01-05T03:49:14Z)
• Beyond Fixed: Variable-Length Denoising for Diffusion Large Language Models [74.15250326312179]
  Diffusion Large Language Models offer efficient parallel generation and capable global modeling.<n>The dominant application ofDLLMs is hindered by the need for a statically predefined generation length.<n>We introduce DAEDAL, a novel training-free denoising strategy that enables Dynamic Adaptive Length Expansion.
  arXiv  Detail & Related papers  (2025-08-01T17:56:07Z)
• Long-Short Alignment for Effective Long-Context Modeling in LLMs [32.13785291956956]
  Large language models (LLMs) have exhibited impressive performance and surprising emergent properties.<n>Length generalization -- the ability to generalize to sequences longer than those seen during training -- is a classical and fundamental problem.<n>We highlight the critical role of textbflong-short alignment -- the consistency of output distributions across sequences of varying lengths.
  arXiv  Detail & Related papers  (2025-06-13T13:25:39Z)
• Zero-Shot Strategies for Length-Controllable Summarization [56.15356055672189]
  Large language models (LLMs) struggle with precise length control, particularly in zero-shot settings. We conduct a comprehensive study evaluating LLMs' length control capabilities across multiple measures and propose practical methods to improve controllability. Our experiments with LLaMA 3 reveal stark differences in length adherence across measures and highlight inherent biases of the model.
  arXiv  Detail & Related papers  (2024-12-31T02:53:27Z)
• Breaking the Context Bottleneck on Long Time Series Forecasting [6.36010639533526]
  Long-term time-series forecasting is essential for planning and decision-making in economics, energy, and transportation. Recent advancements have enhanced the efficiency of these models, but the challenge of effectively leveraging longer sequences persists. We propose the Logsparse Decomposable Multiscaling (LDM) framework for the efficient and effective processing of long sequences.
  arXiv  Detail & Related papers  (2024-12-21T10:29:34Z)
• Length Controlled Generation for Black-box LLMs [70.57649832433451]
  Large language models (LLMs) have demonstrated impressive instruction following capabilities, but struggle to accurately manage the length of generated text. We propose a novel iterative sampling framework for text length control, integrating the Metropolis-Hastings algorithm with an importance sampling acceleration strategy. Our framework achieves almost 100% success rates of length control on Llama3.1 for tasks such as length-controlled abstractive summarization.
  arXiv  Detail & Related papers  (2024-12-19T09:07:38Z)
• Precise Length Control in Large Language Models [1.3654846342364308]
  Large Language Models (LLMs) are increasingly used in production systems.<n>We propose a method to adapt pre-trained decoder-only LLMs for precise control of response length.
  arXiv  Detail & Related papers  (2024-12-16T16:22:27Z)
• LongVQ: Long Sequence Modeling with Vector Quantization on Structured Memory [63.41820940103348]
  Self-attention mechanism's computational cost limits its practicality for long sequences. We propose a new method called LongVQ to compress the global abstraction as a length-fixed codebook. LongVQ effectively maintains dynamic global and local patterns, which helps to complement the lack of long-range dependency issues.
  arXiv  Detail & Related papers  (2024-04-17T08:26:34Z)
• InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory [93.20588235940453]
  In this paper, we introduce a training-free memory-based method, InfLLM. InfLLM stores distant contexts into additional memory units and employs an efficient mechanism to lookup token-relevant units for attention. Even when the sequence length is scaled to $1,024$K, InfLLM still effectively captures long-distance dependencies.
  arXiv  Detail & Related papers  (2024-02-07T06:50:42Z)
• LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models [83.98062659664785]
  Large language models (LLMs) typically train on short text segments (e.g., 4K tokens) due to the quadratic complexity of their Transformer architectures. This work identifies three major factors contributing to this length generalization failure. We propose LM-Infinite, a simple and effective method for enhancing LLMs' capabilities of handling long contexts.
  arXiv  Detail & Related papers  (2023-08-30T16:47:51Z)
• Prompt-Based Length Controlled Generation with Reinforcement Learning [48.49553921757085]
  We propose a prompt-based length control method to achieve high-accuracy length controlled generation. We adopt reinforcement learning with the reward signal given by either trainable or rule-based reward models. Our method significantly improves the accuracy of prompt-based length control for summarization task on popular datasets like CNNDM and NYT.
  arXiv  Detail & Related papers  (2023-08-23T09:43:10Z)
• LLM-Pruner: On the Structural Pruning of Large Language Models [65.02607075556742]
  Large language models (LLMs) have shown remarkable capabilities in language understanding and generation. We tackle the compression of LLMs within the bound of two constraints: being task-agnostic and minimizing the reliance on the original training dataset. Our method, named LLM-Pruner, adopts structural pruning that selectively removes non-critical coupled structures.
  arXiv  Detail & Related papers  (2023-05-19T12:10:53Z)
• Summarization with Precise Length Control [23.688834410051]
  We present a framework to generate summaries with precisely the specified number of tokens or sentences. We jointly train the models to predict the lengths, so our model can generate summaries with optimal length.
  arXiv  Detail & Related papers  (2023-05-09T04:45:24Z)
• LenAtten: An Effective Length Controlling Unit For Text Summarization [5.554982420311913]
  Fixed length summarization aims at generating summaries with a preset number of words or characters. Most recent researches incorporate length information with word embeddings as the input to the recurrent decoding unit. We present an effective length controlling unit Length Attention (LenAtten) to break this trade-off.
  arXiv  Detail & Related papers  (2021-06-01T08:45:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.