Bridging Industrial Expertise and XR with LLM-Powered Conversational Agents

• URL: http://arxiv.org/abs/2504.05527v1
• Date: Mon, 07 Apr 2025 22:02:19 GMT
• Title: Bridging Industrial Expertise and XR with LLM-Powered Conversational Agents
• Authors: Despina Tomkou, George Fatouros, Andreas Andreou, Georgios Makridis, Fotis Liarokapis, Dimitrios Dardanis, Athanasios Kiourtis, John Soldatos, Dimosthenis Kyriazis,
• Abstract summary: This paper introduces a novel integration of Retrieval-Augmented Generation (RAG) enhanced Large Language Models (LLMs) with Extended Reality (XR)<n>The proposed system embeds domain-specific industrial knowledge into XR environments through a natural language interface, enabling hands-free, context-aware expert guidance for workers.
• Score: 2.526333884960815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces a novel integration of Retrieval-Augmented Generation (RAG) enhanced Large Language Models (LLMs) with Extended Reality (XR) technologies to address knowledge transfer challenges in industrial environments. The proposed system embeds domain-specific industrial knowledge into XR environments through a natural language interface, enabling hands-free, context-aware expert guidance for workers. We present the architecture of the proposed system consisting of an LLM Chat Engine with dynamic tool orchestration and an XR application featuring voice-driven interaction. Performance evaluation of various chunking strategies, embedding models, and vector databases reveals that semantic chunking, balanced embedding models, and efficient vector stores deliver optimal performance for industrial knowledge retrieval. The system's potential is demonstrated through early implementation in multiple industrial use cases, including robotic assembly, smart infrastructure maintenance, and aerospace component servicing. Results indicate potential for enhancing training efficiency, remote assistance capabilities, and operational guidance in alignment with Industry 5.0's human-centric and resilient approach to industrial development.

Related papers

• An LLM-enabled Multi-Agent Autonomous Mechatronics Design Framework [49.633199780510864]
  This work proposes a multi-agent autonomous mechatronics design framework, integrating expertise across mechanical design, optimization, electronics, and software engineering. operating primarily through a language-driven workflow, the framework incorporates structured human feedback to ensure robust performance under real-world constraints. A fully functional autonomous vessel was developed with optimized propulsion, cost-effective electronics, and advanced control.
  arXiv  Detail & Related papers  (2025-04-20T16:57:45Z)
• Rethinking industrial artificial intelligence: a unified foundation framework [0.32885740436059047]
  Recent advancements in industrial artificial intelligence (AI) are reshaping the industry by driving smarter manufacturing, predictive maintenance, and intelligent decision-making. Existing approaches often focus primarily on algorithms and models while overlooking the importance of systematically integrating domain knowledge, data, and models. This paper reviews previous research, rethinks the role of industrial AI, and proposes a unified industrial AI foundation framework.
  arXiv  Detail & Related papers  (2025-04-02T15:05:32Z)
• Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap [51.198001060683296]
  Large Language Models (LLMs) offer transformative potential to address transportation challenges.<n>This survey first presents LLM4TR, a novel conceptual framework that systematically categorizes the roles of LLMs in transportation.<n>For each role, our review spans diverse applications, from traffic prediction and autonomous driving to safety analytics and urban mobility optimization.
  arXiv  Detail & Related papers  (2025-03-27T11:56:27Z)
• Towards Agentic Recommender Systems in the Era of Multimodal Large Language Models [75.4890331763196]
  Recent breakthroughs in Large Language Models (LLMs) have led to the emergence of agentic AI systems.<n>LLM-based Agentic RS (LLM-ARS) can offer more interactive, context-aware, and proactive recommendations.
  arXiv  Detail & Related papers  (2025-03-20T22:37:15Z)
• Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimization [62.16747639440893]
  Large language models (LLMs) and their associated technologies advance, particularly in the realms of prompt engineering and agent engineering.<n>Our proposed framework incorporates retrieval-augmented generation (RAG) to enhance the system's ability to acquire domain-specific knowledge and generate solutions.
  arXiv  Detail & Related papers  (2024-08-07T08:43:32Z)
• AI-Powered Immersive Assistance for Interactive Task Execution in Industrial Environments [0.11545092788508222]
  We demonstrate an AI-powered immersive assistance system that supports users in performing complex tasks in industrial environments. Our system leverages a VR environment that resembles a juice mixer setup. This demonstration showcases the potential of our AI-powered assistant to reduce cognitive load, increase productivity, and enhance safety in industrial environments.
  arXiv  Detail & Related papers  (2024-07-12T10:30:45Z)
• Inference Optimization of Foundation Models on AI Accelerators [68.24450520773688]
  Powerful foundation models, including large language models (LLMs), with Transformer architectures have ushered in a new era of Generative AI. As the number of model parameters reaches to hundreds of billions, their deployment incurs prohibitive inference costs and high latency in real-world scenarios. This tutorial offers a comprehensive discussion on complementary inference optimization techniques using AI accelerators.
  arXiv  Detail & Related papers  (2024-07-12T09:24:34Z)
• Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality [28.27036270001756]
  This work designs an autonomous workflow tailored for integrating AI agents seamlessly into extended reality (XR) applications for fine-grained training. We present a demonstration of a multimodal fine-grained training assistant for LEGO brick assembly in a pilot XR environment.
  arXiv  Detail & Related papers  (2024-05-16T14:20:30Z)
• LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application [54.984348122105516]
  Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework synergizes open-world knowledge with collaborative knowledge.<n>We propose an Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework that synergizes open-world knowledge with collaborative knowledge.
  arXiv  Detail & Related papers  (2024-05-07T04:00:30Z)
• The Survey on Multi-Source Data Fusion in Cyber-Physical-Social Systems:Foundational Infrastructure for Industrial Metaverses and Industries 5.0 [31.600740278783242]
  The concept of Industries 5.0 develops, industrial metaverses are expected to operate in parallel with the actual industrial processes. The customized user needs that are hidden in social media data can be discovered by social computing technologies. This work proposes a multi-source-data-fusion-driven operational architecture for industrial metaverses.
  arXiv  Detail & Related papers  (2024-04-11T05:09:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.