Beyond-RAG: Question Identification and Answer Generation in Real-Time Conversations

• URL: http://arxiv.org/abs/2410.10136v1
• Date: Mon, 14 Oct 2024 04:06:22 GMT
• Title: Beyond-RAG: Question Identification and Answer Generation in Real-Time Conversations
• Authors: Garima Agrawal, Sashank Gummuluri, Cosimo Spera,
• Abstract summary: In customer contact centers, human agents often struggle with long average handling times (AHT) We propose a decision support system that can look beyond RAG by first identifying customer questions in real time. If the query matches an FAQ, the system retrieves the answer directly from the FAQ database; otherwise, it generates answers via RAG.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In customer contact centers, human agents often struggle with long average handling times (AHT) due to the need to manually interpret queries and retrieve relevant knowledge base (KB) articles. While retrieval augmented generation (RAG) systems using large language models (LLMs) have been widely adopted in industry to assist with such tasks, RAG faces challenges in real-time conversations, such as inaccurate query formulation and redundant retrieval of frequently asked questions (FAQs). To address these limitations, we propose a decision support system that can look beyond RAG by first identifying customer questions in real time. If the query matches an FAQ, the system retrieves the answer directly from the FAQ database; otherwise, it generates answers via RAG. Our approach reduces reliance on manual queries, providing responses to agents within 2 seconds. Deployed in AI-powered human-agent assist solution at Minerva CQ, this system improves efficiency, reduces AHT, and lowers operational costs. We also introduce an automated LLM-agentic workflow to identify FAQs from historical transcripts when no predefined FAQs exist.

Related papers

• Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent [102.31558123570437]
  Multimodal Retrieval Augmented Generation (mRAG) plays an important role in mitigating the "hallucination" issue inherent in multimodal large language models (MLLMs) We propose the first self-adaptive planning agent for multimodal retrieval, OmniSearch.
  arXiv  Detail & Related papers  (2024-11-05T09:27:21Z)
• AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs [53.6200736559742]
  AGENT-CQ consists of two stages: a generation stage and an evaluation stage. CrowdLLM simulates human crowdsourcing judgments to assess generated questions and answers. Experiments on the ClariQ dataset demonstrate CrowdLLM's effectiveness in evaluating question and answer quality.
  arXiv  Detail & Related papers  (2024-10-25T17:06:27Z)
• RAG-ConfusionQA: A Benchmark for Evaluating LLMs on Confusing Questions [52.33835101586687]
  Conversational AI agents use Retrieval Augmented Generation (RAG) to provide verifiable document-grounded responses to user inquiries. This paper presents a novel synthetic data generation method to efficiently create a diverse set of context-grounded confusing questions from a given document corpus.
  arXiv  Detail & Related papers  (2024-10-18T16:11:29Z)
• Enhancing Retrieval in QA Systems with Derived Feature Association [0.0]
  Retrieval augmented generation (RAG) has become the standard in long context question answering (QA) systems. We propose a novel extension to RAG systems, which we call Retrieval from AI Derived Documents (RAIDD)
  arXiv  Detail & Related papers  (2024-10-02T05:24:49Z)
• RAG based Question-Answering for Contextual Response Prediction System [0.4660328753262075]
  Large Language Models (LLMs) have shown versatility in various Natural Language Processing (NLP) tasks. Retrieval Augmented Generation (RAG) emerges as a promising technique to address this challenge. This paper introduces an end-to-end framework that employs LLMs with RAG capabilities for industry use cases.
  arXiv  Detail & Related papers  (2024-09-05T17:14:23Z)
• CLARINET: Augmenting Language Models to Ask Clarification Questions for Retrieval [52.134133938779776]
  We present CLARINET, a system that asks informative clarification questions by choosing questions whose answers would maximize certainty in the correct candidate. Our approach works by augmenting a large language model (LLM) to condition on a retrieval distribution, finetuning end-to-end to generate the question that would have maximized the rank of the true candidate at each turn.
  arXiv  Detail & Related papers  (2024-04-28T18:21:31Z)
• MFBE: Leveraging Multi-Field Information of FAQs for Efficient Dense Retrieval [1.7403133838762446]
  We propose a bi-encoder-based query-FAQ matching model that leverages multiple combinations of FAQ fields. Our model achieves around 27% and 20% better top-1 accuracy for the FAQ retrieval task on internal and open datasets.
  arXiv  Detail & Related papers  (2023-02-23T12:02:49Z)
• Multi-Tenant Optimization For Few-Shot Task-Oriented FAQ Retrieval [0.0]
  Business-specific Frequently Asked Questions (FAQ) retrieval in task-oriented dialog systems poses unique challenges. We evaluate performance for such Business FAQ using query-Question (q-Q) similarity and few-shot intent detection techniques. We propose a novel approach to scale multi-tenant FAQ applications in real-world context by contrastive fine-tuning of the last layer in sentence Bi-Encoders along with tenant-specific weight switching.
  arXiv  Detail & Related papers  (2023-01-25T10:55:45Z)
• Medical Question Understanding and Answering with Knowledge Grounding and Semantic Self-Supervision [53.692793122749414]
  We introduce a medical question understanding and answering system with knowledge grounding and semantic self-supervision. Our system is a pipeline that first summarizes a long, medical, user-written question, using a supervised summarization loss. The system first matches the summarized user question with an FAQ from a trusted medical knowledge base, and then retrieves a fixed number of relevant sentences from the corresponding answer document.
  arXiv  Detail & Related papers  (2022-09-30T08:20:32Z)
• NoiseQA: Challenge Set Evaluation for User-Centric Question Answering [68.67783808426292]
  We show that components in the pipeline that precede an answering engine can introduce varied and considerable sources of error. We conclude that there is substantial room for progress before QA systems can be effectively deployed.
  arXiv  Detail & Related papers  (2021-02-16T18:35:29Z)
• Towards Automatic Generation of Questions from Long Answers [11.198653485869935]
  We propose a novel evaluation benchmark to assess the performance of existing AQG systems for long-text answers. We empirically demonstrate that the performance of existing AQG methods significantly degrades as the length of the answer increases. Transformer-based methods outperform other existing AQG methods on long answers in terms of automatic as well as human evaluation.
  arXiv  Detail & Related papers  (2020-04-10T16:45:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.