Towards autonomous system: flexible modular production system enhanced with large language model agents

• URL: http://arxiv.org/abs/2304.14721v4
• Date: Mon, 24 Jul 2023 09:49:55 GMT
• Title: Towards autonomous system: flexible modular production system enhanced with large language model agents
• Authors: Yuchen Xia, Manthan Shenoy, Nasser Jazdi, Michael Weyrich
• Abstract summary: We present a novel framework that combines large language models (LLMs), digital twins and industrial automation system. We demonstrate how our implemented prototype can handle un-predefined tasks, plan a production process, and execute the operations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we present a novel framework that combines large language models (LLMs), digital twins and industrial automation system to enable intelligent planning and control of production processes. We retrofit the automation system for a modular production facility and create executable control interfaces of fine-granular functionalities and coarse-granular skills. Low-level functionalities are executed by automation components, and high-level skills are performed by automation modules. Subsequently, a digital twin system is developed, registering these interfaces and containing additional descriptive information about the production system. Based on the retrofitted automation system and the created digital twins, LLM-agents are designed to interpret descriptive information in the digital twins and control the physical system through service interfaces. These LLM-agents serve as intelligent agents on different levels within an automation system, enabling autonomous planning and control of flexible production. Given a task instruction as input, the LLM-agents orchestrate a sequence of atomic functionalities and skills to accomplish the task. We demonstrate how our implemented prototype can handle un-predefined tasks, plan a production process, and execute the operations. This research highlights the potential of integrating LLMs into industrial automation systems in the context of smart factory for more agile, flexible, and adaptive production processes, while it also underscores the critical insights and limitations for future work. Demos at: https://github.com/YuchenXia/GPT4IndustrialAutomation

Related papers

• AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML [56.565200973244146]
  Automated machine learning (AutoML) accelerates AI development by automating tasks in the development pipeline. Recent works have started exploiting large language models (LLM) to lessen such burden. This paper proposes AutoML-Agent, a novel multi-agent framework tailored for full-pipeline AutoML.
  arXiv  Detail & Related papers  (2024-10-03T20:01:09Z)
• Control Industrial Automation System with Large Language Models [2.2369578015657954]
  This paper introduces a framework for integrating large language models with industrial automation systems. At the core of the framework are an agent system designed for industrial tasks, a structured prompting method, and an event-driven information modeling mechanism. Our contribution includes a formal system design, proof-of-concept implementation, and a method for generating task-specific datasets.
  arXiv  Detail & Related papers  (2024-09-26T16:19:37Z)
• Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? [73.81908518992161]
  We introduce Spider2-V, the first multimodal agent benchmark focusing on professional data science and engineering. Spider2-V features real-world tasks in authentic computer environments and incorporating 20 enterprise-level professional applications. These tasks evaluate the ability of a multimodal agent to perform data-related tasks by writing code and managing the GUI in enterprise data software systems.
  arXiv  Detail & Related papers  (2024-07-15T17:54:37Z)
• Incorporating Large Language Models into Production Systems for Enhanced Task Automation and Flexibility [2.3999111269325266]
  This paper introduces a novel approach to integrating large language model (LLM) agents into automated production systems. We organize production operations within a hierarchical framework based on the automation pyramid. This allows for a scalable and flexible foundation for orchestrating production processes.
  arXiv  Detail & Related papers  (2024-07-11T14:34:43Z)
• ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning [74.58666091522198]
  We present a framework for intuitive robot programming by non-experts. We leverage natural language prompts and contextual information from the Robot Operating System (ROS) Our system integrates large language models (LLMs), enabling non-experts to articulate task requirements to the system through a chat interface.
  arXiv  Detail & Related papers  (2024-06-28T08:28:38Z)
• A Cyber Manufacturing IoT System for Adaptive Machine Learning Model Deployment by Interactive Causality Enabled Self-Labeling [0.0]
  This paper proposes the AdaptIoT system, comprised of an end-to-end data streaming pipeline, ML service integration, and an automated self-labeling service. The self-labeling service consists of causal knowledge bases and automated full-cycle self-labeling to adapt multiple ML models simultaneously. A field demonstration of a self-labeling adaptive ML application is conducted with a makerspace and shows reliable performance.
  arXiv  Detail & Related papers  (2024-04-09T03:10:45Z)
• AutoMMLab: Automatically Generating Deployable Models from Language Instructions for Computer Vision Tasks [39.71649832548044]
  AutoMMLab is a general-purpose LLM-empowered AutoML system that follows user's language instructions. The proposed AutoMMLab system effectively employs LLMs as the bridge to connect AutoML and OpenMMLab community. Experiments show that our AutoMMLab system is versatile and covers a wide range of mainstream tasks.
  arXiv  Detail & Related papers  (2024-02-23T14:38:19Z)
• TaskBench: Benchmarking Large Language Models for Task Automation [82.2932794189585]
  We introduce TaskBench, a framework to evaluate the capability of large language models (LLMs) in task automation. Specifically, task decomposition, tool selection, and parameter prediction are assessed. Our approach combines automated construction with rigorous human verification, ensuring high consistency with human evaluation.
  arXiv  Detail & Related papers  (2023-11-30T18:02:44Z)
• ProAgent: From Robotic Process Automation to Agentic Process Automation [87.0555252338361]
  Large Language Models (LLMs) have emerged human-like intelligence. This paper introduces Agentic Process Automation (APA), a groundbreaking automation paradigm using LLM-based agents for advanced automation. We then instantiate ProAgent, an agent designed to craft from human instructions and make intricate decisions by coordinating specialized agents.
  arXiv  Detail & Related papers  (2023-11-02T14:32:16Z)
• OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System [85.8338446357469]
  We introduce OmniForce, a human-centered AutoML system that yields both human-assisted ML and ML-assisted human techniques. We show how OmniForce can put an AutoML system into practice and build adaptive AI in open-environment scenarios.
  arXiv  Detail & Related papers  (2023-03-01T13:35:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.