Quantized Embedding Vectors for Controllable Diffusion Language Models
- URL: http://arxiv.org/abs/2402.10107v1
- Date: Thu, 15 Feb 2024 17:02:48 GMT
- Title: Quantized Embedding Vectors for Controllable Diffusion Language Models
- Authors: Cheng Kang, Xinye Chen, Yong Hu, Daniel Novak
- Abstract summary: Quantized Embedding Controllable Diffusion Language Model improves controllability, portability, and inference speed of language models.
QE-CDLM builds upon the recent successful controllable DLMs by remodeling the task-specific embedding space via quantization.
- Score: 1.3287140837287783
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Improving the controllability, portability, and inference speed of diffusion
language models (DLMs) is a key challenge in natural language generation. While
recent research has shown significant success in complex text generation with
language models, the memory and computational power are still very demanding
and fall short of expectations, which naturally results in low portability and
instability for the models. To mitigate these issues, numerous well-established
methods were proposed for neural network quantization. To further enhance their
portability of independent deployment as well as improve their stability
evaluated by language perplexity, we propose a novel approach called the
Quantized Embedding Controllable Diffusion Language Model (QE-CDLM). QE-CDLM
builds upon the recent successful controllable DLMs by remodeling the
task-specific embedding space via quantization. This leads to a gradient-based
controller for the generation tasks, and more stable intermediate latent
variables are obtained, which naturally brings in an accelerated convergence as
well as better controllability. Additionally, the adaption fine-tuning method
is employed to reduce tunable weights. Experimental results on five challenging
fine-grained control tasks demonstrate that QE-CDLM compares favorably to
existing methods in terms of quality and feasibility, achieving better
perplexity and lightweight fine-tuning.
Related papers
- Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization [50.485788083202124]
Reinforcement Learning (RL) plays a crucial role in aligning large language models with human preferences and improving their ability to perform complex tasks.
We introduce Direct Q-function Optimization (DQO), which formulates the response generation process as a Markov Decision Process (MDP) and utilizes the soft actor-critic (SAC) framework to optimize a Q-function directly parameterized by the language model.
Experimental results on two math problem-solving datasets, GSM8K and MATH, demonstrate that DQO outperforms previous methods, establishing it as a promising offline reinforcement learning approach for aligning language models.
arXiv Detail & Related papers (2024-10-11T23:29:20Z) - Attribute Controlled Fine-tuning for Large Language Models: A Case Study on Detoxification [76.14641982122696]
We propose a constraint learning schema for fine-tuning Large Language Models (LLMs) with attribute control.
We show that our approach leads to an LLM that produces fewer inappropriate responses while achieving competitive performance on benchmarks and a toxicity detection task.
arXiv Detail & Related papers (2024-10-07T23:38:58Z) - Reference Trustable Decoding: A Training-Free Augmentation Paradigm for Large Language Models [79.41139393080736]
Large language models (LLMs) have rapidly advanced and demonstrated impressive capabilities.
We propose Reference Trustable Decoding (RTD), a paradigm that allows models to quickly adapt to new tasks without fine-tuning.
arXiv Detail & Related papers (2024-09-30T10:48:20Z) - Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation [70.22782550540714]
Quantization-aware Scale LeArning method based on multimodal Warmup, termed QSLAW.
We introduce a Quantization-aware Scale LeArning method based on multimodal Warmup, termed QSLAW.
arXiv Detail & Related papers (2024-08-07T12:42:09Z) - Unveiling LLM Mechanisms Through Neural ODEs and Control Theory [3.4039202831583903]
This study uses Neural Ordinary Differential Equations to unravel the intricate relationships between inputs and outputs in Large Language Models (LLMs)
Neural ODEs play a pivotal role in this investigation by providing a dynamic model that captures the continuous evolution of data within the LLMs.
robust control mechanisms are applied to strategically adjust the model's outputs, ensuring they not only maintain high quality and reliability but also adhere to specific performance criteria.
arXiv Detail & Related papers (2024-06-23T22:56:34Z) - PID Control-Based Self-Healing to Improve the Robustness of Large Language Models [23.418411870842178]
Minor perturbations can significantly reduce the performance of well-trained language models.
We construct a computationally efficient self-healing process to correct undesired model behavior.
The proposed PID control-based self-healing is a low cost framework that improves the robustness of pre-trained large language models.
arXiv Detail & Related papers (2024-03-31T23:46:51Z) - LlaMaVAE: Guiding Large Language Model Generation via Continuous Latent
Sentence Spaces [1.529963465178546]
We present LlaMaVAE, which combines expressive encoder and decoder models (sentenceT5 and LlaMA) with a VAE architecture to provide better text generation control to large language models (LLMs)
Experimental results reveal that LlaMaVAE can outperform the previous state-of-the-art VAE language model, Optimus, across various tasks.
arXiv Detail & Related papers (2023-12-20T17:25:23Z) - Tailoring Language Generation Models under Total Variation Distance [55.89964205594829]
The standard paradigm of neural language generation adopts maximum likelihood estimation (MLE) as the optimizing method.
We develop practical bounds to apply it to language generation.
We introduce the TaiLr objective that balances the tradeoff of estimating TVD.
arXiv Detail & Related papers (2023-02-26T16:32:52Z) - Diffusion-LM Improves Controllable Text Generation [80.50044830018442]
Controlling the behavior of language models (LMs) without re-training is a major open problem in natural language generation.
We develop a new non-autoregressive language model based on continuous diffusions that we call Diffusion-LM.
We demonstrate successful control of Diffusion-LM for six challenging fine-grained control tasks, significantly outperforming prior work.
arXiv Detail & Related papers (2022-05-27T20:12:09Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.