ECO Decoding: Entropy-Based Control for Controllability and Fluency in Controllable Dialogue Generation

• URL: http://arxiv.org/abs/2511.01568v1
• Date: Mon, 03 Nov 2025 13:35:37 GMT
• Title: ECO Decoding: Entropy-Based Control for Controllability and Fluency in Controllable Dialogue Generation
• Authors: Seungmin Shin, Dooyoung Kim, Youngjoong Ko,
• Abstract summary: We propose ECO decoding, which dynamically adjusts the control strength at each generation step according to the model's entropy.<n>Experiments on the DailyDialog and MultiWOZ datasets demonstrate that ECO decoding consistently improves controllability while maintaining fluency and grammaticality.
• Score: 20.658872192907705
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Controllable Dialogue Generation (CDG) enables chatbots to generate responses with desired attributes, and weighted decoding methods have achieved significant success in the CDG task. However, using a fixed constant value to manage the bias of attribute probabilities makes it challenging to find an ideal control strength that satisfies both controllability and fluency. To address this issue, we propose ECO decoding (Entropy-based COntrol), which dynamically adjusts the control strength at each generation step according to the model's entropy in both the language model and attribute classifier probability distributions. Experiments on the DailyDialog and MultiWOZ datasets demonstrate that ECO decoding consistently improves controllability while maintaining fluency and grammaticality, outperforming prior decoding methods across various models and settings. Furthermore, ECO decoding alleviates probability interpolation issues in multi-attribute generation and consequently demonstrates strong performance in both single and multi-attribute scenarios.

Related papers

• CARD: Towards Conditional Design of Multi-agent Topological Structures [83.18278008173746]
  CARD (Conditional Agentic Graph Designer) is a conditional graph-generation framework that instantiates AMACP, a protocol for adaptive multi-agent communication.<n> CARD produces communication structures that are both effective and resilient to shifts in model capability or resource availability.
  arXiv  Detail & Related papers  (2026-03-01T13:02:36Z)
• Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs [49.66344956133349]
  Reasoning capacity shapes both inference-time performance and reinforcement learning (RL) training for large (vision-) language models.<n>This paper proposes Reasoning Palette, a novel latent-modulation framework that endows the model with a latent variable for strategic contextualization.
  arXiv  Detail & Related papers  (2025-12-19T03:32:53Z)
• Auxiliary-Hyperparameter-Free Sampling: Entropy Equilibrium for Text Generation [20.748382951054563]
  Token sampling strategies influence text generation quality in large language models (LLMs)<n>We present Entropy Equilibrium Sampling (EES), an auxiliary hyper parameter-free approach inspired by information theory.<n>EES consistently performs well across temperature settings, delivering competitive accuracy and coherence while maintaining diversity.
  arXiv  Detail & Related papers  (2025-11-30T08:58:08Z)
• A high-capacity linguistic steganography based on entropy-driven rank-token mapping [81.29800498695899]
  Linguistic steganography enables covert communication through embedding secret messages into innocuous texts.<n>Traditional modification-based methods introduce detectable anomalies, while retrieval-based strategies suffer from low embedding capacity.<n>We propose an entropy-driven framework called RTMStega that integrates rank-based adaptive coding and context-aware decompression with normalized entropy.
  arXiv  Detail & Related papers  (2025-10-27T06:02:47Z)
• Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning [65.20602712957725]
  Caco is a novel framework that automates the synthesis of high-quality, verifiable, and diverse instruction-CoT reasoning data.<n>Our work establishes a paradigm for building self-sustaining, trustworthy reasoning systems without human intervention.
  arXiv  Detail & Related papers  (2025-10-05T07:59:24Z)
• SCALAR: Scale-wise Controllable Visual Autoregressive Learning [15.775596699630633]
  We present SCALAR, a controllable generation method based on Visual Autoregressive ( VAR)<n>We leverage a pretrained image encoder to extract semantic control signal encodings, which are projected into scale-specific representations and injected into the corresponding layers of the VAR backbone.<n>Building on SCALAR, we develop SCALAR-Uni, a unified extension that aligns multiple control modalities into a shared latent space, supporting flexible multi-conditional guidance in a single model.
  arXiv  Detail & Related papers  (2025-07-26T13:23:08Z)
• ControlVAR: Exploring Controllable Visual Autoregressive Modeling [48.66209303617063]
  Conditional visual generation has witnessed remarkable progress with the advent of diffusion models (DMs) Challenges such as expensive computational cost, high inference latency, and difficulties of integration with large language models (LLMs) have necessitated exploring alternatives to DMs. This paper introduces Controlmore, a novel framework that explores pixel-level controls in visual autoregressive modeling for flexible and efficient conditional generation.
  arXiv  Detail & Related papers  (2024-06-14T06:35:33Z)
• Quantized Embedding Vectors for Controllable Diffusion Language Models [1.3287140837287783]
  Quantized Embedding Controllable Diffusion Language Model improves controllability, portability, and inference speed of language models. QE-CDLM builds upon the recent successful controllable DLMs by remodeling the task-specific embedding space via quantization.
  arXiv  Detail & Related papers  (2024-02-15T17:02:48Z)
• Controllable Text Generation with Residual Memory Transformer [4.9329649616940205]
  We propose a non-intrusive, lightweight control plugin to accompany the generation of CLM at arbitrary time steps. The proposed plugin, namely Residual Memory Transformer (RMT), has an encoder-decoder setup, which can accept any types of control conditions. Extensive experiments are carried out on various control tasks, in the form of both automatic and human evaluations.
  arXiv  Detail & Related papers  (2023-09-28T08:13:33Z)
• Semantic Space Grounded Weighted Decoding for Multi-Attribute Controllable Dialogue Generation [41.23970507903113]
  We propose a novel framework called DASC that possesses strong controllability with a weighted decoding paradigm. Generation with multiple attributes is then intuitively implemented with an utterance of multiple attribute embeddings. Experiments show that DASC can achieve high control accuracy in generation task with the simultaneous control of 3 aspects.
  arXiv  Detail & Related papers  (2023-05-04T13:35:27Z)
• Is Disentanglement enough? On Latent Representations for Controllable Music Generation [78.8942067357231]
  In the absence of a strong generative decoder, disentanglement does not necessarily imply controllability. The structure of the latent space with respect to the VAE-decoder plays an important role in boosting the ability of a generative model to manipulate different attributes.
  arXiv  Detail & Related papers  (2021-08-01T18:37:43Z)
• Control, Generate, Augment: A Scalable Framework for Multi-Attribute Text Generation [22.70189685469752]
  We introduce CGA, a conditional VAE architecture, to control, generate, and augment text. We show the value of the individual model components in an ablation study. We show high quality, diversity and attribute control in the generated sentences through a series of automatic and human assessments.
  arXiv  Detail & Related papers  (2020-04-30T17:31:16Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.