Efficient and Accurate Memorable Conversation Model using DPO based on sLLM

• URL: http://arxiv.org/abs/2407.06537v2
• Date: Tue, 27 Aug 2024 04:43:59 GMT
• Title: Efficient and Accurate Memorable Conversation Model using DPO based on sLLM
• Abstract summary: This paper presents a conversation model that efficiently manages memory as sessions progress. Our model using DPO algorithm shows an improvement about 0.0591 of BERTScore in memory accuracy. Our model demonstrates efficiency not only in terms of accuracy but also in resource utilization.
• Score: 0.1285666675142425
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In multi-session dialog system, it is essential to continuously update the memory as the session progresses. Simply accumulating memory can make it difficult to focus on the content of the conversation for inference due to the limited input sentence size. Therefore, efficient and accurate conversation model that is capable of managing memory to reflect the conversation history continuously is necessary. This paper presents a conversation model that efficiently manages memory as sessions progress and incorporates this into the model to reflect the conversation history accurately with 3 methodologies: SFT, DPO and DPO with SFT model. Our model using DPO algorithm shows an improvement about 0.0591 of BERTScore in memory accuracy, and the rate of responses reflecting the memory increased as well. Also, response generation performance enhanced about 4.292 in fluency, 3.935 in coherence, and 2.896 in consistency. This paper describes a training method that yields better performance than models with more than twice the parameter size, even when the model size is smaller. Thus, our model demonstrates efficiency not only in terms of accuracy but also in resource utilization.

Related papers

• Memory-Augmented Query Intent Understanding for Efficient Chat-based Image Retrieval [74.93546115702155]
  Chat-based image retrieval allows the human-interactive system to iteratively clarify and refine user intent through multi-round dialogue.<n>The key challenge in this task lies in dynamically understanding and updating the user's query intent across dialogue rounds.<n>This paper proposes a novel and efficient memory-based user intent updating framework for the chat-based image retrieval task.
  arXiv  Detail & Related papers  (2026-05-17T10:17:41Z)
• Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents [68.84161689205779]
  Temporal Semantic Memory (TSM) is a memory framework that models semantic time for point-wise memory.<n>TSM consistently outperforms existing methods and achieves up to 12.2% absolute improvement in accuracy.
  arXiv  Detail & Related papers  (2026-01-12T12:24:44Z)
• Pre-Storage Reasoning for Episodic Memory: Shifting Inference Burden to Memory for Personalized Dialogue [13.558061425427688]
  PREMem is a novel approach that shifts complex reasoning processes from inference to memory construction.<n>It creates enriched representations while reducing computational demands during interactions.<n> Experiments show significant performance improvements across all model sizes.
  arXiv  Detail & Related papers  (2025-09-13T15:18:08Z)
• LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues [55.332261548836485]
  LoopServe is an adaptive dual-phase inference framework for large language models in multi-turn dialogues.<n>It consistently achieves superior effectiveness compared to existing baselines.<n>It significantly accelerates inference across a wide range of long-context dialogue tasks.
  arXiv  Detail & Related papers  (2025-07-18T06:12:08Z)
• Memory Layers at Scale [67.00854080570979]
  This work takes memory layers beyond proof-of-concept, proving their utility at contemporary scale. On downstream tasks, language models augmented with our improved memory layer outperform dense models with more than twice the budget, as well as mixture-of-expert models when matched for both compute and parameters. We provide a fully parallelizable memory layer implementation, demonstrating scaling laws with up to 128B memory parameters, pretrained to 1 trillion tokens, comparing to base models with up to 8B parameters.
  arXiv  Detail & Related papers  (2024-12-12T23:56:57Z)
• Self-Updatable Large Language Models with Parameter Integration [21.742149718161716]
  Small-scale experiences, such as interactions with surrounding objects, require frequent integration in large language models. Current methods embed experiences within model parameters using continual learning, model editing, or knowledge distillation techniques. We propose SELF-PARAM, which embeds experiences directly into model parameters and ensures near-optimal efficacy and long-term retention.
  arXiv  Detail & Related papers  (2024-10-01T08:18:17Z)
• Compressed Context Memory For Online Language Model Interaction [39.72054168889216]
  This paper presents a context key/value compression method for Transformer language models in online scenarios. As the context lengthens, the attention process demands increasing memory and computations, which in turn reduces the throughput of the language model. We propose a compressed context memory system that continually compresses the accumulating attention key/value pairs into a compact memory space.
  arXiv  Detail & Related papers  (2023-12-06T10:50:43Z)
• SSP: Self-Supervised Post-training for Conversational Search [63.28684982954115]
  We propose fullmodel (model) which is a new post-training paradigm with three self-supervised tasks to efficiently initialize the conversational search model. To verify the effectiveness of our proposed method, we apply the conversational encoder post-trained by model on the conversational search task using two benchmark datasets: CAsT-19 and CAsT-20.
  arXiv  Detail & Related papers  (2023-07-02T13:36:36Z)
• UniMC: A Unified Framework for Long-Term Memory Conversation via Relevance Representation Learning [15.313416157905685]
  We propose a Unified framework for Long-term Memory Conversations (UniMC) We decompose the main task into three subtasks based on probability graphs. Each subtask involves learning a representation for calculating the relevance between the query and memory.
  arXiv  Detail & Related papers  (2023-06-18T12:30:50Z)
• Enhancing Large Language Model with Self-Controlled Memory Framework [56.38025154501917]
  Large Language Models (LLMs) are constrained by their inability to process lengthy inputs, resulting in the loss of critical historical information. We propose the Self-Controlled Memory (SCM) framework to enhance the ability of LLMs to maintain long-term memory and recall relevant information.
  arXiv  Detail & Related papers  (2023-04-26T07:25:31Z)
• Stateful Memory-Augmented Transformers for Efficient Dialogue Modeling [69.31802246621963]
  We propose a novel memory-augmented transformer that is compatible with existing pre-trained encoder-decoder models. By incorporating a separate memory module alongside the pre-trained transformer, the model can effectively interchange information between the memory states and the current input context.
  arXiv  Detail & Related papers  (2022-09-15T22:37:22Z)
• A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning [56.450090618578]
  Class-Incremental Learning (CIL) aims to train a model with limited memory size to meet this requirement. We show that when counting the model size into the total budget and comparing methods with aligned memory size, saving models do not consistently work. We propose a simple yet effective baseline, denoted as MEMO for Memory-efficient Expandable MOdel.
  arXiv  Detail & Related papers  (2022-05-26T08:24:01Z)
• A Bi-Encoder LSTM Model For Learning Unstructured Dialogs [0.0]
  This paper presents a Long Short Term Memory (LSTM) based architecture that learns unstructured multi-turn dialogs. We show that our model achieves 0.8%, 1.0% and 0.3% higher accuracy for Recall@1, Recall@2 and Recall@5 respectively than the benchmark model.
  arXiv  Detail & Related papers  (2021-04-25T21:37:35Z)
• Plug-and-Play Conversational Models [62.77150879036442]
  We introduce an approach that does not require further computation at decoding time, while also does not require any fine-tuning of a large language model. We demonstrate, through extensive automatic and human evaluation, a high degree of control over the generated conversational responses with regard to multiple desired attributes.
  arXiv  Detail & Related papers  (2020-10-09T03:17:51Z)
• Modality-Balanced Models for Visual Dialogue [102.35406085738325]
  The Visual Dialog task requires a model to exploit both image and conversational context information to generate the next response to the dialogue. We show that previous joint-modality (history and image) models over-rely on and are more prone to memorizing the dialogue history. We present methods for this integration of the two models, via ensemble and consensus dropout fusion with shared parameters.
  arXiv  Detail & Related papers  (2020-01-17T14:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.