Thread Detection and Response Generation using Transformers with Prompt Optimisation

• URL: http://arxiv.org/abs/2403.05931v1
• Date: Sat, 9 Mar 2024 14:50:20 GMT
• Title: Thread Detection and Response Generation using Transformers with Prompt Optimisation
• Authors: Kevin Joshua T, Arnav Agarwal, Shriya Sanjay, Yash Sarda, John Sahaya Rani Alex, Saurav Gupta, Sushant Kumar, Vishwanath Kamath
• Abstract summary: This paper develops an end-to-end model that identifies threads and prioritises their response generation based on the importance. The model achieves up to 10x speed improvement, while generating more coherent results compared to existing models.
• Score: 5.335657953493376
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Conversational systems are crucial for human-computer interaction, managing complex dialogues by identifying threads and prioritising responses. This is especially vital in multi-party conversations, where precise identification of threads and strategic response prioritisation ensure efficient dialogue management. To address these challenges an end-to-end model that identifies threads and prioritises their response generation based on the importance was developed, involving a systematic decomposition of the problem into discrete components - thread detection, prioritisation, and performance optimisation which was meticulously analysed and optimised. These refined components seamlessly integrate into a unified framework, in conversational systems. Llama2 7b is used due to its high level of generalisation but the system can be updated with any open source Large Language Model(LLM). The computational capabilities of the Llama2 model was augmented by using fine tuning methods and strategic prompting techniques to optimise the model's performance, reducing computational time and increasing the accuracy of the model. The model achieves up to 10x speed improvement, while generating more coherent results compared to existing models.

Related papers

• Automatically Learning Hybrid Digital Twins of Dynamical Systems [56.69628749813084]
  Digital Twins (DTs) simulate the states and temporal dynamics of real-world systems. DTs often struggle to generalize to unseen conditions in data-scarce settings. In this paper, we propose an evolutionary algorithm ($textbfHDTwinGen$) to autonomously propose, evaluate, and optimize HDTwins.
  arXiv  Detail & Related papers  (2024-10-31T07:28:22Z)
• Reliable LLM-based User Simulator for Task-Oriented Dialogue Systems [2.788542465279969]
  This paper introduces DAUS, a Domain-Aware User Simulator. We fine-tune DAUS on real examples of task-oriented dialogues. Results on two relevant benchmarks showcase significant improvements in terms of user goal fulfillment.
  arXiv  Detail & Related papers  (2024-02-20T20:57:47Z)
• PICK: Polished & Informed Candidate Scoring for Knowledge-Grounded Dialogue Systems [59.1250765143521]
  Current knowledge-grounded dialogue systems often fail to align the generated responses with human-preferred qualities. We propose Polished & Informed Candidate Scoring (PICK), a generation re-scoring framework. We demonstrate the effectiveness of PICK in generating responses that are more faithful while keeping them relevant to the dialogue history.
  arXiv  Detail & Related papers  (2023-09-19T08:27:09Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluation [54.66399120084227]
  Language models trained on large-scale corpora can generate remarkably fluent results in open-domain dialogue. For the persona-based dialogue generation task, consistency and coherence are great challenges for language models. A two-stage SimOAP strategy is proposed, i.e., over-sampling and post-evaluation.
  arXiv  Detail & Related papers  (2023-05-18T17:23:00Z)
• Evaluating and Improving Context Attention Distribution on Multi-Turn Response Generation using Self-Contained Distractions [0.18275108630751835]
  We focus on an essential component of multi-turn generation-based conversational agents: context attention distribution. To improve performance on this component, we propose an optimization strategy that employs self-contained distractions. Our experiments on the Ubuntu chatlogs dataset show that models with comparable perplexity can be distinguished by their ability on context attention distribution.
  arXiv  Detail & Related papers  (2022-11-09T15:12:20Z)
• Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning [27.04857039060308]
  We propose to combine the best of both worlds to build a retrieval system. We employ a fast bi-encoder to replace the traditional feature-based pre-retrieval model. We train the pre-retrieval model and the re-ranking model at the same time via mutual learning.
  arXiv  Detail & Related papers  (2021-10-01T01:32:33Z)
• Enhancing Dialogue Generation via Multi-Level Contrastive Learning [57.005432249952406]
  We propose a multi-level contrastive learning paradigm to model the fine-grained quality of the responses with respect to the query. A Rank-aware (RC) network is designed to construct the multi-level contrastive optimization objectives. We build a Knowledge Inference (KI) component to capture the keyword knowledge from the reference during training and exploit such information to encourage the generation of informative words.
  arXiv  Detail & Related papers  (2020-09-19T02:41:04Z)
• Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues [88.73739515457116]
  We introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination. We jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner. Experiment results indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection.
  arXiv  Detail & Related papers  (2020-09-14T08:44:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.