Should RAG Chatbots Forget Unimportant Conversations? Exploring Importance and Forgetting with Psychological Insights

• URL: http://arxiv.org/abs/2409.12524v1
• Date: Thu, 19 Sep 2024 07:39:22 GMT
• Title: Should RAG Chatbots Forget Unimportant Conversations? Exploring Importance and Forgetting with Psychological Insights
• Authors: Ryuichi Sumida, Koji Inoue, Tatsuya Kawahara,
• Abstract summary: We propose LUFY, a method that focuses on emotionally arousing memories and retains less than 10% of the conversation. The results demonstrate that prioritizing arousing memories while forgetting the majority of the conversation significantly enhances user experience.
• Score: 21.68243297242355
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Retrieval-Augmented Generation (RAG) has shown promise in enhancing long-term conversations, the increasing memory load as conversations progress degrades retrieval accuracy. Drawing on psychological insights, we propose LUFY, a simple yet effective method that focuses on emotionally arousing memories and retains less than 10% of the conversation. In the user experiment, participants interacted with three types of RAG chatbots, each for 2 hours over 4 sessions, marking the most extensive assessment of a chatbot's long-term capabilities to date -- more than four times longer than any existing benchmark. The results demonstrate that prioritizing arousing memories while forgetting the majority of the conversation significantly enhances user experience. This study pushes the frontier of long-term conversations and highlights the importance of forgetting unimportant parts of conversations. Code and Dataset: https://github.com/ryuichi-sumida/LUFY

Related papers

• LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory [68.97819665784442]
  This paper introduces LongMemEval, a benchmark designed to evaluate five core long-term memory abilities of chat assistants. LongMemEval presents a significant challenge to existing long-term memory systems. We present a unified framework that breaks down the long-term memory design into four design choices.
  arXiv  Detail & Related papers  (2024-10-14T17:59:44Z)
• Introducing MeMo: A Multimodal Dataset for Memory Modelling in Multiparty Conversations [1.8896253910986929]
  MeMo corpus is the first dataset annotated with participants' memory retention reports. It integrates validated behavioural and perceptual measures, audio, video, and multimodal annotations. This paper aims to pave the way for future research in conversational memory modelling for intelligent system development.
  arXiv  Detail & Related papers  (2024-09-07T16:09:36Z)
• Self-Directed Turing Test for Large Language Models [56.64615470513102]
  The Turing test examines whether AIs can exhibit human-like behaviour in natural language conversations. Traditional Turing tests adopt a rigid dialogue format where each participant sends only one message each time. This paper proposes the Self-Directed Turing Test, which extends the original test with a burst dialogue format.
  arXiv  Detail & Related papers  (2024-08-19T09:57:28Z)
• Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models [75.98775135321355]
  Given a long conversation, large language models (LLMs) fail to recall past information and tend to generate inconsistent responses. We propose to generate summaries/ memory using large language models (LLMs) to enhance long-term memory ability.
  arXiv  Detail & Related papers  (2023-08-29T04:59:53Z)
• SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue Agents [72.42049370297849]
  SpokenWOZ is a large-scale speech-text dataset for spoken TOD. Cross-turn slot and reasoning slot detection are new challenges for SpokenWOZ.
  arXiv  Detail & Related papers  (2023-05-22T13:47:51Z)
• AFEC: A Knowledge Graph Capturing Social Intelligence in Casual Conversations [7.390960543869484]
  This paper introduces AFEC, an automatically curated knowledge graph based on people's day-to-day casual conversations. For this body of knowledge to be comprehensive and meaningful, we curated a large-scale corpus from the r/CasualConversation SubReddit.
  arXiv  Detail & Related papers  (2022-05-22T15:19:12Z)
• CoMPM: Context Modeling with Speaker's Pre-trained Memory Tracking for Emotion Recognition in Conversation [0.0]
  We introduce a context embedding module (CoM) combined with a pre-trained memory module (PM) We show that the pre-trained memory significantly improves the final accuracy of emotion recognition. We experimented on both the multi-party datasets (MELD, EmoryNLP) and the dyadic-party datasets (IEMOCAP, DailyDialog)
  arXiv  Detail & Related papers  (2021-08-26T07:45:09Z)
• Intelligent Conversational Android ERICA Applied to Attentive Listening and Job Interview [41.789773897391605]
  We have developed an intelligent conversational android ERICA. We set up several social interaction tasks for ERICA, including attentive listening, job interview, and speed dating. It has been evaluated with 40 senior people, engaged in conversation of 5-7 minutes without a conversation breakdown.
  arXiv  Detail & Related papers  (2021-05-02T06:37:23Z)
• Exploiting Unsupervised Data for Emotion Recognition in Conversations [76.01690906995286]
  Emotion Recognition in Conversations (ERC) aims to predict the emotional state of speakers in conversations. The available supervised data for the ERC task is limited. We propose a novel approach to leverage unsupervised conversation data.
  arXiv  Detail & Related papers  (2020-10-02T13:28:47Z)
• Pchatbot: A Large-Scale Dataset for Personalized Chatbot [49.16746174238548]
  We introduce Pchatbot, a large-scale dialogue dataset that contains two subsets collected from Weibo and Judicial forums respectively. To adapt the raw dataset to dialogue systems, we elaborately normalize the raw dataset via processes such as anonymization. The scale of Pchatbot is significantly larger than existing Chinese datasets, which might benefit the data-driven models.
  arXiv  Detail & Related papers  (2020-09-28T12:49:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.