Hallucinations and Truth: A Comprehensive Accuracy Evaluation of RAG, LoRA and DoRA

• URL: http://arxiv.org/abs/2502.10497v1
• Date: Fri, 14 Feb 2025 17:38:25 GMT
• Title: Hallucinations and Truth: A Comprehensive Accuracy Evaluation of RAG, LoRA and DoRA
• Authors: Mohammad Baqar, Rajat Khanda,
• Abstract summary: Recent advancements in Generative AI have significantly improved the efficiency and adaptability of natural language processing (NLP) systems. This paper presents a large-scale empirical evaluation of RAG, LoRA, and DoRA, with model fine-tuning and generation performance assessed on 20,000 FAQ-based queries. DoRA achieves the highest accuracy (90.1%), relevance score (0.88), and lowest latency (110 ms per query), outperforming both LoRA and RAG in real-world, domain-specific generative AI applications.
• Score: 0.0
• License:
• Abstract: Recent advancements in Generative AI have significantly improved the efficiency and adaptability of natural language processing (NLP) systems, particularly through Retrieval-Augmented Generation (RAG), Low-Rank Adaptation (LoRA), and Weight-Decomposed Low-Rank Adaptation (DoRA). RAG integrates external knowledge to enhance factual consistency in generative outputs, while LoRA enables parameter-efficient fine-tuning of large language models (LLMs). DoRA further refines this process by optimizing fine-tuning through adaptive parameter ranking and domain-aware weight adjustments, improving learning efficiency while maintaining inference performance. This paper presents a large-scale empirical evaluation of RAG, LoRA, and DoRA, with model fine-tuning and generation performance assessed on 20,000 FAQ-based queries, while the knowledge base spans 400,000 entries. The study analyzes key performance metrics such as accuracy, relevance, and inference latency. Experimental results demonstrate that DoRA achieves the highest accuracy (90.1%), relevance score (0.88), and lowest latency (110 ms per query), outperforming both LoRA and RAG in real-world, domain-specific generative AI applications. Furthermore, this study examines the trade-offs between fine-tuning efficiency, computational cost, and real-time adaptability across different models. Findings highlight RAG's effectiveness in knowledge grounding, LoRA's cost-efficient domain adaptation, and DoRA's ability to balance fine-tuning efficiency with model precision. These insights provide practical guidance for deploying AI-driven generative systems in accuracy-critical domains such as healthcare, finance, and legal services, ensuring scalability, reliability, and optimal performance in dynamic environments.

Related papers

• BeamLoRA: Beam-Constraint Low-Rank Adaptation [51.52097743781401]
  Low-Rank Adaptation (LoRA) has been widely adopted as one of the most effective parameter-efficient fine-tuning methods. We propose BeamLoRA, which conceptualizes each LoRA module as a beam where each rank naturally corresponds to a potential sub-solution.
  arXiv  Detail & Related papers  (2025-02-19T10:33:22Z)
• GoRA: Gradient-driven Adaptive Low Rank Adaptation [11.937225965088963]
  Low-Rank Adaptation (LoRA) is a crucial method for efficiently fine-tuning large language models. We introduce GoRA (Gradient-driven Adaptive Low Rank Adaptation), which adaptively assigns ranks and initializes weights for low-rank adapters. GoRA significantly improves performance while preserving the high usability and efficiency of LoRA.
  arXiv  Detail & Related papers  (2025-02-13T10:33:58Z)
• HRP: High-Rank Preheating for Superior LoRA Initialization [58.3319586613105]
  High-Rank Preheating (HRP) is proposed to fine-tune Low-Rank Adaptation (LoRA) HRP significantly enhances LoRA's generalization effectiveness across various models and tasks.
  arXiv  Detail & Related papers  (2025-02-11T17:59:35Z)
• DeepRAG: Thinking to Retrieval Step by Step for Large Language Models [92.87532210660456]
  We propose DeepRAG, a framework that models retrieval-augmented reasoning as a Markov Decision Process (MDP) By iteratively decomposing queries, DeepRAG dynamically determines whether to retrieve external knowledge or rely on parametric reasoning at each step. Experiments show that DeepRAG improves retrieval efficiency while improving answer accuracy by 21.99%, demonstrating its effectiveness in optimizing retrieval-augmented reasoning.
  arXiv  Detail & Related papers  (2025-02-03T08:22:45Z)
• Dynamic Adaptation of LoRA Fine-Tuning for Efficient and Task-Specific Optimization of Large Language Models [0.7421845364041001]
  This paper presents a novel methodology of fine-tuning for large language models-dynamic LoRA. It adds dynamic adaptation mechanisms to improve efficiency and performance. The efficiency of the dynamic LoRA was validated in experiments on benchmark datasets.
  arXiv  Detail & Related papers  (2025-01-24T18:54:14Z)
• The Efficiency vs. Accuracy Trade-off: Optimizing RAG-Enhanced LLM Recommender Systems Using Multi-Head Early Exit [46.37267466656765]
  This paper presents an optimization framework that combines Retrieval-Augmented Generation (RAG) with an innovative multi-head early exit architecture. Our experiments demonstrate how this architecture effectively decreases time without sacrificing the accuracy needed for reliable recommendation delivery.
  arXiv  Detail & Related papers  (2025-01-04T03:26:46Z)
• Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks [1.236974227340167]
  This study proposes a large language model optimization method based on the improved LoRA fine-tuning algorithm. We fine-tune the large language model through a low-rank adaptation strategy, which significantly reduces the consumption of computing resources. The improved LoRA algorithm shows significant improvements in accuracy, F1 score, and MCC compared with traditional models.
  arXiv  Detail & Related papers  (2024-12-25T01:10:25Z)
• GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning [2.7446241148152253]
  Fine-tuning large language models (LLMs) is computationally intensive because it requires updating all parameters. Low-Rank Adaptation (LoRA) improves efficiency by modifying only a subset of weights but introduces a trade-off between expressivity and computational cost. We propose Geometric Low-Rank Adaptation (GeLoRA), a novel framework that computes the intrinsic dimensionality of hidden state representations to adaptively select LoRA ranks.
  arXiv  Detail & Related papers  (2024-12-12T13:04:54Z)
• Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs [75.11449420928139]
  Fine-tuning Large Language Models (LLMs) has become a crucial technique for adapting pre-trained models to downstream tasks. Low-Rank Adaptation (LoRA) has emerged as a promising solution, but there exists a gap between the practical performance of low-rank adaptations and its theoretical optimum. We propose eXtreme Gradient Boosting LoRA, a novel framework that bridges this gap by leveraging the power of ensemble learning.
  arXiv  Detail & Related papers  (2024-10-25T17:07:13Z)
• Leveraging the Domain Adaptation of Retrieval Augmented Generation Models for Question Answering and Reducing Hallucination [3.879723687090678]
  Retrieval Augmented Generation Model stands out to be highly effective on downstream applications like Question Answering. Recently, RAG-end2end model further optimized the architecture and achieved notable performance improvements on domain adaptation. In this paper, we investigated the performance of diverse RAG and RAG-like architectures through domain adaptation.
  arXiv  Detail & Related papers  (2024-10-23T11:32:46Z)
• Sparse Low-rank Adaptation of Pre-trained Language Models [79.74094517030035]
  We introduce sparse low-rank adaptation (SoRA) that enables dynamic adjustments to the intrinsic rank during the adaptation process. Our approach strengthens the representation power of LoRA by initializing it with a higher rank, while efficiently taming a temporarily increased number of parameters. Our experimental results demonstrate that SoRA can outperform other baselines even with 70% retained parameters and 70% training time.
  arXiv  Detail & Related papers  (2023-11-20T11:56:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.