ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation

• URL: http://arxiv.org/abs/2502.08178v1
• Date: Wed, 12 Feb 2025 07:32:48 GMT
• Title: ParetoRAG: Leveraging Sentence-Context Attention for Robust and Efficient Retrieval-Augmented Generation
• Authors: Ruobing Yao, Yifei Zhang, Shuang Song, Yuhua Liu, Neng Gao, Chenyang Tu,
• Abstract summary: We present an unsupervised framework that optimize Retrieval-Augmented Generation (RAG) systems. By decomposing paragraphs into sentences, we dynamically re-weighting core content while preserving contextual coherence. This framework has been validated across various datasets, Large Language Models (LLMs), and retrievers.
• Score: 8.223134723149753
• License:
• Abstract: While Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by incorporating external knowledge, they still face persistent challenges in retrieval inefficiency and the inability of LLMs to filter out irrelevant information. We present ParetoRAG, an unsupervised framework that optimizes RAG systems through sentence-level refinement guided by the Pareto principle. By decomposing paragraphs into sentences and dynamically re-weighting core content while preserving contextual coherence, ParetoRAG achieves dual improvements in both retrieval precision and generation quality without requiring additional training or API resources. This framework has been empirically validated across various datasets, LLMs, and retrievers.

Related papers

• EAGER-LLM: Enhancing Large Language Models as Recommenders through Exogenous Behavior-Semantic Integration [60.47645731801866]
  Large language models (LLMs) are increasingly leveraged as foundational backbones in advanced recommender systems. LLMs are pre-trained linguistic semantics but learn collaborative semantics from scratch via the llm-Backbone. We propose EAGER-LLM, a decoder-only generative recommendation framework that integrates endogenous and endogenous behavioral and semantic information in a non-intrusive manner.
  arXiv  Detail & Related papers  (2025-02-20T17:01:57Z)
• DeepRAG: Thinking to Retrieval Step by Step for Large Language Models [92.87532210660456]
  We propose DeepRAG, a framework that models retrieval-augmented reasoning as a Markov Decision Process (MDP) By iteratively decomposing queries, DeepRAG dynamically determines whether to retrieve external knowledge or rely on parametric reasoning at each step. Experiments show that DeepRAG improves retrieval efficiency while improving answer accuracy by 21.99%, demonstrating its effectiveness in optimizing retrieval-augmented reasoning.
  arXiv  Detail & Related papers  (2025-02-03T08:22:45Z)
• RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects [12.5122702720856]
  We propose Robust Fine-Tuning (RbFT) to enhance the resilience of large language models against retrieval defects. Experimental results demonstrate that RbFT significantly improves the robustness of RAG systems across diverse retrieval conditions.
  arXiv  Detail & Related papers  (2025-01-30T14:15:09Z)
• FRAG: A Flexible Modular Framework for Retrieval-Augmented Generation based on Knowledge Graphs [17.477161619378332]
  We propose a novel flexible modular KG-RAG framework, termed FRAG, which synergizes the advantages of both approaches. By using the query text instead of the Knowledge Graph, FRAG improves retrieval quality while maintaining flexibility.
  arXiv  Detail & Related papers  (2025-01-17T05:19:14Z)
• TrustRAG: Enhancing Robustness and Trustworthiness in RAG [31.231916859341865]
  TrustRAG is a framework that systematically filters compromised and irrelevant contents before they are retrieved for generation. TrustRAG delivers substantial improvements in retrieval accuracy, efficiency, and attack resistance compared to existing approaches.
  arXiv  Detail & Related papers  (2025-01-01T15:57:34Z)
• Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generation [38.80878966092216]
  Recent Retrieval Augmented Generation (RAG) aims to enhance Large Language Models (LLMs) We propose the chain-of-verification (CoV-RAG) to enhance the external retrieval correctness and internal generation consistency.
  arXiv  Detail & Related papers  (2024-10-08T08:34:54Z)
• RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework [69.4501863547618]
  This paper introduces RAGEval, a framework designed to assess RAG systems across diverse scenarios. With a focus on factual accuracy, we propose three novel metrics Completeness, Hallucination, and Irrelevance. Experimental results show that RAGEval outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples.
  arXiv  Detail & Related papers  (2024-08-02T13:35:11Z)
• Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation [64.7982176398485]
  Retrieval-augmented generation (RAG) has demonstrated effectiveness in mitigating the hallucination problem of large language models (LLMs) We propose DPA-RAG, a universal framework designed to align diverse knowledge preferences within RAG systems.
  arXiv  Detail & Related papers  (2024-06-26T18:26:53Z)
• Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation [128.01050030936028]
  We propose an information refinement training method named InFO-RAG. InFO-RAG is low-cost and general across various tasks. It improves the performance of LLaMA2 by an average of 9.39% relative points.
  arXiv  Detail & Related papers  (2024-02-28T08:24:38Z)
• Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection [74.51523859064802]
  We introduce a new framework called Self-Reflective Retrieval-Augmented Generation (Self-RAG) Self-RAG enhances an LM's quality and factuality through retrieval and self-reflection. It significantly outperforms state-of-the-art LLMs and retrieval-augmented models on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-10-17T18:18:32Z)
• RAGAS: Automated Evaluation of Retrieval Augmented Generation [25.402461447140823]
  RAGAs is a framework for evaluation of Retrieval Augmented Generation pipelines. RAG systems are composed of a retrieval and an LLM based generation module.
  arXiv  Detail & Related papers  (2023-09-26T19:23:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.