Enabling Real-Time Conversations with Minimal Training Costs

• URL: http://arxiv.org/abs/2409.11727v1
• Date: Wed, 18 Sep 2024 06:27:26 GMT
• Title: Enabling Real-Time Conversations with Minimal Training Costs
• Authors: Wang Xu, Shuo Wang, Weilin Zhao, Xu Han, Yukun Yan, Yudi Zhang, Zhe Tao, Zhiyuan Liu, Wanxiang Che,
• Abstract summary: This paper presents a new duplex decoding approach that enhances large language models with duplex ability, requiring minimal training. Experimental results indicate that our proposed method significantly enhances the naturalness and human-likeness of user-AI interactions with minimal training costs.
• Score: 61.80370154101649
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have demonstrated the ability to improve human efficiency through conversational interactions. Conventional LLM-powered dialogue systems, operating on a turn-based paradigm, preclude real-time interaction during response generation. To address this limitation, researchers have proposed duplex models. These models can dynamically adapt to user input, facilitating real-time interactive feedback. However, these methods typically require substantial computational resources to acquire the ability. To reduce overhead, this paper presents a new duplex decoding approach that enhances LLMs with duplex ability, requiring minimal additional training. Specifically, our method employs parallel decoding of queries and responses in conversations, effectively implementing a channel-division-multiplexing decoding strategy. Experimental results indicate that our proposed method significantly enhances the naturalness and human-likeness of user-AI interactions with minimal training costs.

Related papers

• RA-BLIP: Multimodal Adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training [55.54020926284334]
  Multimodal Large Language Models (MLLMs) have recently received substantial interest, which shows their emerging potential as general-purpose models for various vision-language tasks. Retrieval augmentation techniques have proven to be effective plugins for both LLMs and MLLMs. In this study, we propose multimodal adaptive Retrieval-Augmented Bootstrapping Language-Image Pre-training (RA-BLIP), a novel retrieval-augmented framework for various MLLMs.
  arXiv  Detail & Related papers  (2024-10-18T03:45:19Z)
• Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models [66.24055500785657]
  Traditional turn-based chat systems prevent users from verbally interacting with system while it is generating responses. To overcome these limitations, we adapt existing LLMs to listen users while generating output and provide users with instant feedback. We build a dataset consisting of alternating time slices of queries and responses as well as covering typical feedback types in instantaneous interactions.
  arXiv  Detail & Related papers  (2024-06-22T03:20:10Z)
• Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training [33.57497419019826]
  Action-Based Contrastive Self-Training allows for sample-efficient dialogue policy learning in multi-turn conversation. ACT demonstrates substantial conversation modeling improvements over standard approaches to supervised fine-tuning and DPO.
  arXiv  Detail & Related papers  (2024-05-31T22:44:48Z)
• RLIF: Interactive Imitation Learning as Reinforcement Learning [56.997263135104504]
  We show how off-policy reinforcement learning can enable improved performance under assumptions that are similar but potentially even more practical than those of interactive imitation learning. Our proposed method uses reinforcement learning with user intervention signals themselves as rewards. This relaxes the assumption that intervening experts in interactive imitation learning should be near-optimal and enables the algorithm to learn behaviors that improve over the potential suboptimal human expert.
  arXiv  Detail & Related papers  (2023-11-21T21:05:21Z)
• Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations [70.7884839812069]
  Large language models (LLMs) have emerged as powerful and general solutions to many natural language tasks. However, many of the most important applications of language generation are interactive, where an agent has to talk to a person to reach a desired outcome. In this work, we explore a new method for adapting LLMs with RL for such goal-directed dialogue.
  arXiv  Detail & Related papers  (2023-11-09T18:45:16Z)
• Dialog Action-Aware Transformer for Dialog Policy Learning [22.262659702998892]
  We propose to make full use of the plain text knowledge from the pre-trained language model to accelerate the RL agent's learning speed. Specifically, we design a dialog action-aware transformer encoder (DaTrans) which integrates a new fine-tuning procedure named masked last action task. DaTrans is further optimized in an RL setting with ongoing interactions and evolves through exploration in the dialog action space toward maximizing long-term accumulated rewards.
  arXiv  Detail & Related papers  (2023-09-05T13:47:25Z)
• Replicating Complex Dialogue Policy of Humans via Offline Imitation Learning with Supervised Regularization [7.151589223349882]
  Policy learning (PL) is a module of a task-oriented dialogue system that trains an agent to make actions in each dialogue turn. Both supervised learning (SL) and reinforcement learning (RL) frameworks cannot imitate humans well. This study proposed an offline imitation learning model that learns policy from real dialogue datasets.
  arXiv  Detail & Related papers  (2023-05-06T09:27:58Z)
• Deep RL with Hierarchical Action Exploration for Dialogue Generation [0.0]
  This paper presents theoretical analysis and experiments that reveal the performance of the dialogue policy is positively correlated with the sampling size. We introduce a novel dual-granularity Q-function that explores the most promising response category to intervene in the sampling process. Our algorithm exhibits both explainability and controllability and generates responses with higher expected rewards.
  arXiv  Detail & Related papers  (2023-03-22T09:29:22Z)
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
  We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
  arXiv  Detail & Related papers  (2021-06-09T14:10:50Z)
• Sample-Efficient Model-based Actor-Critic for an Interactive Dialogue Task [27.896714528986855]
  We present a model-based reinforcement learning for an interactive dialogue task. We build on commonly used actor-critic methods, adding an environment model and planner that augments a learning agent to learn. Our results show that, on a simulation that mimics the interactive task our algorithm requires 70 times fewer samples, compared to the baseline of commonly used model-free algorithm.
  arXiv  Detail & Related papers  (2020-04-28T17:00:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.