Retrieval-Augmented Generation in Industry: An Interview Study on Use Cases, Requirements, Challenges, and Evaluation

• URL: http://arxiv.org/abs/2508.14066v1
• Date: Mon, 11 Aug 2025 09:40:54 GMT
• Title: Retrieval-Augmented Generation in Industry: An Interview Study on Use Cases, Requirements, Challenges, and Evaluation
• Authors: Lorenz Brehme, Benedikt Dornauer, Thomas Ströhle, Maximilian Ehrhart, Ruth Breu,
• Abstract summary: Retrieval-Augmented Generation (RAG) is a rapidly evolving field within AI.<n>There is a significant lack of research on its practical application in industrial contexts.<n>Our study investigates how companies apply RAG in practice.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-Augmented Generation (RAG) is a well-established and rapidly evolving field within AI that enhances the outputs of large language models by integrating relevant information retrieved from external knowledge sources. While industry adoption of RAG is now beginning, there is a significant lack of research on its practical application in industrial contexts. To address this gap, we conducted a semistructured interview study with 13 industry practitioners to explore the current state of RAG adoption in real-world settings. Our study investigates how companies apply RAG in practice, providing (1) an overview of industry use cases, (2) a consolidated list of system requirements, (3) key challenges and lessons learned from practical experiences, and (4) an analysis of current industry evaluation methods. Our main findings show that current RAG applications are mostly limited to domain-specific QA tasks, with systems still in prototype stages; industry requirements focus primarily on data protection, security, and quality, while issues such as ethics, bias, and scalability receive less attention; data preprocessing remains a key challenge, and system evaluation is predominantly conducted by humans rather than automated methods.

Related papers

• Aligning Academia with Industry: An Empirical Study of Industrial Needs and Academic Capabilities in AI-Driven Software Engineering [45.09204791294318]
  The rapid advancement of large language models (LLMs) is fundamentally reshaping software engineering (SE)<n>While top-tier SE venues continue to show sustained or emerging focus on areas like automated testing and program repair, the alignment of these academic advances with real industrial needs remains unclear.<n>This study aims to refocus academic attention on these important yet under-explored problems and to guide future SE research toward greater industrial impact.
  arXiv  Detail & Related papers  (2025-12-17T07:29:18Z)
• Let the Barbarians In: How AI Can Accelerate Systems Performance Research [80.43506848683633]
  We term this iterative cycle of generation, evaluation, and refinement AI-Driven Research for Systems.<n>We demonstrate that ADRS-generated solutions can match or even outperform human state-of-the-art designs.
  arXiv  Detail & Related papers  (2025-12-16T18:51:23Z)
• Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents [63.03252293761656]
  This paper systematically reviews the technologies, applications, and evaluation methods of industry agents based on large language models (LLMs)<n>We examine the three key technological pillars that support the advancement of agent capabilities: Memory, Planning, and Tool Use.<n>We provide an overview of the application of industry agents in real-world domains such as digital engineering, scientific discovery, embodied intelligence, collaborative business execution, and complex system simulation.
  arXiv  Detail & Related papers  (2025-10-20T12:46:55Z)
• A Systematic Review of Key Retrieval-Augmented Generation (RAG) Systems: Progress, Gaps, and Future Directions [1.4931265249949528]
  Retrieval-Augmented Generation (RAG) is a major advancement in natural language processing (NLP)<n>RAG combines large language models (LLMs) with information retrieval systems to enhance factual grounding, accuracy, and contextual relevance.<n>This paper presents a systematic review of RAG, tracing its evolution from early developments in open domain question answering to recent state-of-the-art implementations.
  arXiv  Detail & Related papers  (2025-07-25T03:05:46Z)
• Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation [4.448709087838503]
  Retrieval-Augmented Generation (RAG) systems are emerging as a key approach for grounding Large Language Models (LLMs) in external knowledge.<n>This paper presents five domain-specific RAG applications developed for real-world scenarios across governance, cybersecurity, agriculture, industrial research, and medical diagnostics.
  arXiv  Detail & Related papers  (2025-06-25T22:40:00Z)
• Beyond Academic Benchmarks: Critical Analysis and Best Practices for Visual Industrial Anomaly Detection [40.174488947319645]
  Anomaly detection (AD) is essential for automating visual inspection in manufacturing.<n>This paper makes three key contributions: (1) we demonstrate the importance of real-world datasets and establish benchmarks using actual production data; (2) we provide a fair comparison of existing SOTA methods across diverse tasks by utilizing metrics that are valuable for practical applications; and (3) we present a comprehensive analysis of recent advancements in this field by discussing important challenges and new perspectives for bridging the academia-industry gap.
  arXiv  Detail & Related papers  (2025-03-30T14:11:46Z)
• State of play and future directions in industrial computer vision AI standards [6.5545889890947295]
  Artificial Intelligence (AI) and Deep Learning (DL) have resulted into corresponding remarkable progress in the field of Computer Vision (CV)<n>This study investigates the current state of play regarding the development of industrial computer vision AI standards.
  arXiv  Detail & Related papers  (2025-03-04T14:46:34Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• Anomaly Detection for Industrial Applications, Its Challenges, Solutions, and Future Directions: A Review [4.139740414165092]
  Anomaly detection from images captured using camera sensors is one of the mainstream applications at the industrial level.<n>Traditional anomaly detection workflow is based on a manual inspection by human operators.<n>Recent vision-based approaches can automatically extract, process, and interpret features using computer vision.
  arXiv  Detail & Related papers  (2025-01-20T07:24:39Z)
• Trustworthiness in Retrieval-Augmented Generation Systems: A Survey [59.26328612791924]
  Retrieval-Augmented Generation (RAG) has quickly grown into a pivotal paradigm in the development of Large Language Models (LLMs) We propose a unified framework that assesses the trustworthiness of RAG systems across six key dimensions: factuality, robustness, fairness, transparency, accountability, and privacy.
  arXiv  Detail & Related papers  (2024-09-16T09:06:44Z)
• A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models [71.25225058845324]
  Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation. Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge. RA-LLMs have emerged to harness external and authoritative knowledge bases, rather than relying on the model's internal knowledge.
  arXiv  Detail & Related papers  (2024-05-10T02:48:45Z)
• The Technological Emergence of AutoML: A Survey of Performant Software and Applications in the Context of Industry [72.10607978091492]
  Automated/Autonomous Machine Learning (AutoML/AutonoML) is a relatively young field. This review makes two primary contributions to knowledge around this topic. It provides the most up-to-date and comprehensive survey of existing AutoML tools, both open-source and commercial.
  arXiv  Detail & Related papers  (2022-11-08T10:42:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.