ComfyUI-Copilot: An Intelligent Assistant for Automated Workflow Development

• URL: http://arxiv.org/abs/2506.05010v1
• Date: Thu, 05 Jun 2025 13:20:50 GMT
• Title: ComfyUI-Copilot: An Intelligent Assistant for Automated Workflow Development
• Authors: Zhenran Xu, Xue Yang, Yiyu Wang, Qingli Hu, Zijiao Wu, Longyue Wang, Weihua Luo, Kaifu Zhang, Baotian Hu, Min Zhang,
• Abstract summary: ComfyUI-Copilot is a large language model-powered plugin for ComfyUI.<n>It offers intelligent node and model recommendations, along with automated one-click workflow construction.<n>We validate the effectiveness of ComfyUI-Copilot through both offline quantitative evaluations and online user feedback.
• Score: 45.78818581469798
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce ComfyUI-Copilot, a large language model-powered plugin designed to enhance the usability and efficiency of ComfyUI, an open-source platform for AI-driven art creation. Despite its flexibility and user-friendly interface, ComfyUI can present challenges to newcomers, including limited documentation, model misconfigurations, and the complexity of workflow design. ComfyUI-Copilot addresses these challenges by offering intelligent node and model recommendations, along with automated one-click workflow construction. At its core, the system employs a hierarchical multi-agent framework comprising a central assistant agent for task delegation and specialized worker agents for different usages, supported by our curated ComfyUI knowledge bases to streamline debugging and deployment. We validate the effectiveness of ComfyUI-Copilot through both offline quantitative evaluations and online user feedback, showing that it accurately recommends nodes and accelerates workflow development. Additionally, use cases illustrate that ComfyUI-Copilot lowers entry barriers for beginners and enhances workflow efficiency for experienced users. The ComfyUI-Copilot installation package and a demo video are available at https://github.com/AIDC-AI/ComfyUI-Copilot.

Related papers

• A Human Centric Requirements Engineering Framework for Assessing Github Copilot Output [0.0]
  GitHub Copilot introduces new challenges in how these software tools address human needs.<n>I analyzed GitHub Copilot's interaction with users through its chat interface.<n>I established a human-centered requirements framework with clear metrics to evaluate these qualities.
  arXiv  Detail & Related papers  (2025-08-05T21:33:23Z)
• Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows [66.1850490474361]
  We conduct the first academic study to explore developer interactions with coding agents.<n>We evaluate two leading copilot and agentic coding assistants, GitHub Copilot and OpenHands.<n>Our results show agents have the potential to assist developers in ways that surpass copilots.
  arXiv  Detail & Related papers  (2025-07-10T20:12:54Z)
• ComfyUI-R1: Exploring Reasoning Models for Workflow Generation [49.09090292453288]
  We introduce ComfyUI-R1, first large reasoning model for automated workflow generation.<n>We construct long chain-of-thought (CoT) reasoning data, including node selection, workflow planning, and code-level workflow.<n> Experiments show that our 7B- parameter model achieves a 97% format validity rate, along with high pass rate, node-level and graph-level F1 scores.
  arXiv  Detail & Related papers  (2025-06-11T14:35:15Z)
• AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuning [82.42421823672954]
  AgentCPM-GUI is built for robust and efficient on-device GUI interaction.<n>Our training pipeline includes grounding-aware pre-training to enhance perception.<n>AgentCPM-GUI achieves state-of-the-art performance on five public benchmarks.
  arXiv  Detail & Related papers  (2025-06-02T07:30:29Z)
• Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy [5.985777189633703]
  AI copilots represent a new generation of AI-powered systems designed to assist users in complex, context-rich tasks.<n>Central to this personalization is preference optimization: the system's ability to detect, interpret, and align with individual user preferences.<n>This survey examines how user preferences are operationalized in AI copilots.
  arXiv  Detail & Related papers  (2025-05-28T02:52:39Z)
• Do It For Me vs. Do It With Me: Investigating User Perceptions of Different Paradigms of Automation in Copilots for Feature-Rich Software [9.881955481813465]
  Large Language Model (LLM)-based in-application assistants, or copilots, can automate software tasks.<n>We investigated two automation paradigms by designing and implementing a fully automated copilot and a semi-automated copilot.<n>GuidedCopilot automates trivial steps while offering step-by-step visual guidance.
  arXiv  Detail & Related papers  (2025-04-22T03:11:10Z)
• ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generation [71.31634636156384]
  We introduce ComfyGPT, the first self-optimizing multi-agent system designed to generate ComfyUI based on task descriptions automatically.<n> ComfyGPT comprises four specialized agents: ReformatAgent, FlowAgent, RefineAgent, and ExecuteAgent.<n> FlowDataset is a large-scale dataset containing 13,571 workflow-description pairs, and FlowBench is a benchmark for evaluating workflow generation systems.
  arXiv  Detail & Related papers  (2025-03-22T06:48:50Z)
• ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systems [80.69865295743149]
  This work attempts to study using LLM-based agents to design collaborative AI systems autonomously.<n>Based on ComfyBench, we develop ComfyAgent, a framework that empowers agents to autonomously design collaborative AI systems by generating.<n>While ComfyAgent achieves a comparable resolve rate to o1-preview and significantly surpasses other agents on ComfyBench, ComfyAgent has resolved only 15% of creative tasks.
  arXiv  Detail & Related papers  (2024-09-02T17:44:10Z)
• CarDreamer: Open-Source Learning Platform for World Model based Autonomous Driving [25.49856190295859]
  World model (WM) based reinforcement learning (RL) has emerged as a promising approach by learning and predicting the complex dynamics of various environments. There does not exist an accessible platform for training and testing such algorithms in sophisticated driving environments. We introduce CarDreamer, the first open-source learning platform designed specifically for developing WM based autonomous driving algorithms.
  arXiv  Detail & Related papers  (2024-05-15T05:57:20Z)
• OS-Copilot: Towards Generalist Computer Agents with Self-Improvement [48.29860831901484]
  We introduce OS-Copilot, a framework to build generalist agents capable of interfacing with comprehensive elements in an operating system (OS) We use OS-Copilot to create FRIDAY, a self-improving embodied agent for automating general computer tasks. On GAIA, a general AI assistants benchmark, FRIDAY outperforms previous methods by 35%, showcasing strong generalization to unseen applications via accumulated skills from previous tasks.
  arXiv  Detail & Related papers  (2024-02-12T07:29:22Z)
• Exploring the Problems, their Causes and Solutions of AI Pair Programming: A Study on GitHub and Stack Overflow [6.724815667295355]
  GitHub Copilot, the AI programmer pair, utilize machine learning models trained on a large corpus of code snippets to generate code suggestions. Despite its popularity in software development, there is limited empirical evidence on the actual experiences of practitioners who work with Copilot. We collected data from 473 GitHub issues, 706 GitHub discussions, and 142 Stack Overflow posts.
  arXiv  Detail & Related papers  (2023-11-02T06:24:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.