Towards autonomous system: flexible modular production system enhanced
with large language model agents
- URL: http://arxiv.org/abs/2304.14721v4
- Date: Mon, 24 Jul 2023 09:49:55 GMT
- Title: Towards autonomous system: flexible modular production system enhanced
with large language model agents
- Authors: Yuchen Xia, Manthan Shenoy, Nasser Jazdi, Michael Weyrich
- Abstract summary: We present a novel framework that combines large language models (LLMs), digital twins and industrial automation system.
We demonstrate how our implemented prototype can handle un-predefined tasks, plan a production process, and execute the operations.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In this paper, we present a novel framework that combines large language
models (LLMs), digital twins and industrial automation system to enable
intelligent planning and control of production processes. We retrofit the
automation system for a modular production facility and create executable
control interfaces of fine-granular functionalities and coarse-granular skills.
Low-level functionalities are executed by automation components, and high-level
skills are performed by automation modules. Subsequently, a digital twin system
is developed, registering these interfaces and containing additional
descriptive information about the production system. Based on the retrofitted
automation system and the created digital twins, LLM-agents are designed to
interpret descriptive information in the digital twins and control the physical
system through service interfaces. These LLM-agents serve as intelligent agents
on different levels within an automation system, enabling autonomous planning
and control of flexible production. Given a task instruction as input, the
LLM-agents orchestrate a sequence of atomic functionalities and skills to
accomplish the task. We demonstrate how our implemented prototype can handle
un-predefined tasks, plan a production process, and execute the operations.
This research highlights the potential of integrating LLMs into industrial
automation systems in the context of smart factory for more agile, flexible,
and adaptive production processes, while it also underscores the critical
insights and limitations for future work. Demos at:
https://github.com/YuchenXia/GPT4IndustrialAutomation
Related papers
- Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? [73.81908518992161]
We introduce Spider2-V, the first multimodal agent benchmark focusing on professional data science and engineering.
Spider2-V features real-world tasks in authentic computer environments and incorporating 20 enterprise-level professional applications.
These tasks evaluate the ability of a multimodal agent to perform data-related tasks by writing code and managing the GUI in enterprise data software systems.
arXiv Detail & Related papers (2024-07-15T17:54:37Z) - Incorporating Large Language Models into Production Systems for Enhanced Task Automation and Flexibility [2.3999111269325266]
This paper introduces a novel approach to integrating large language model (LLM) agents into automated production systems.
We organize production operations within a hierarchical framework based on the automation pyramid.
This allows for a scalable and flexible foundation for orchestrating production processes.
arXiv Detail & Related papers (2024-07-11T14:34:43Z) - ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning [74.58666091522198]
We present a framework for intuitive robot programming by non-experts.
We leverage natural language prompts and contextual information from the Robot Operating System (ROS)
Our system integrates large language models (LLMs), enabling non-experts to articulate task requirements to the system through a chat interface.
arXiv Detail & Related papers (2024-06-28T08:28:38Z) - From Language Models to Practical Self-Improving Computer Agents [0.8547032097715571]
We develop a methodology to create AI computer agents that can carry out diverse computer tasks and self-improve.
We prompt an LLM agent to augment itself with retrieval, internet search, web navigation, and text editor capabilities.
The agent effectively uses these various tools to solve problems including automated software development and web-based tasks.
arXiv Detail & Related papers (2024-04-18T07:50:10Z) - A Cyber Manufacturing IoT System for Adaptive Machine Learning Model Deployment by Interactive Causality Enabled Self-Labeling [0.0]
This paper proposes the AdaptIoT system, comprised of an end-to-end data streaming pipeline, ML service integration, and an automated self-labeling service.
The self-labeling service consists of causal knowledge bases and automated full-cycle self-labeling to adapt multiple ML models simultaneously.
A field demonstration of a self-labeling adaptive ML application is conducted with a makerspace and shows reliable performance.
arXiv Detail & Related papers (2024-04-09T03:10:45Z) - AutoMMLab: Automatically Generating Deployable Models from Language
Instructions for Computer Vision Tasks [39.71649832548044]
AutoMMLab is a general-purpose LLM-empowered AutoML system that follows user's language instructions.
The proposed AutoMMLab system effectively employs LLMs as the bridge to connect AutoML and OpenMMLab community.
Experiments show that our AutoMMLab system is versatile and covers a wide range of mainstream tasks.
arXiv Detail & Related papers (2024-02-23T14:38:19Z) - LLMind: Orchestrating AI and IoT with LLM for Complex Task Execution [20.186752447895994]
We present LLMind, an AI agent framework that enables effective collaboration among IoT devices for executing complex tasks.
Inspired by the functional specialization theory of the brain, our framework integrates an LLM with domain-specific AI modules, enhancing its capabilities.
arXiv Detail & Related papers (2023-12-14T14:57:58Z) - TaskBench: Benchmarking Large Language Models for Task Automation [85.3879908356586]
We introduce TaskBench to evaluate the capability of large language models in task automation.
To generate high-quality evaluation datasets, we introduce the concept of Tool Graph.
We also propose TaskEval to evaluate the capability of LLMs from different aspects, including task decomposition, tool invocation, and parameter prediction.
arXiv Detail & Related papers (2023-11-30T18:02:44Z) - ProAgent: From Robotic Process Automation to Agentic Process Automation [87.0555252338361]
Large Language Models (LLMs) have emerged human-like intelligence.
This paper introduces Agentic Process Automation (APA), a groundbreaking automation paradigm using LLM-based agents for advanced automation.
We then instantiate ProAgent, an agent designed to craft from human instructions and make intricate decisions by coordinating specialized agents.
arXiv Detail & Related papers (2023-11-02T14:32:16Z) - A Graphical Modeling Language for Artificial Intelligence Applications
in Automation Systems [69.50862982117127]
An interdisciplinary graphical modeling language that enables the modeling of an AI application as an overall system comprehensible to all disciplines does not yet exist.
This paper presents a graphical modeling language that enables consistent and understandable modeling of AI applications in automation systems at system level.
arXiv Detail & Related papers (2023-06-20T12:06:41Z) - OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge
Collaborative AutoML System [85.8338446357469]
We introduce OmniForce, a human-centered AutoML system that yields both human-assisted ML and ML-assisted human techniques.
We show how OmniForce can put an AutoML system into practice and build adaptive AI in open-environment scenarios.
arXiv Detail & Related papers (2023-03-01T13:35:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.