JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents

• URL: http://arxiv.org/abs/2607.23588v1
• Date: Sun, 26 Jul 2026 10:36:24 GMT
• Title: JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents
• Abstract summary: JarvisHub is a canvas-native creative agent harness for long-horizon multimodal creation.<n>It represents multimodal artifacts, dependencies, versions, and feedback as typed canvas nodes and links.<n>This design moves creative agents beyond isolated tool use toward sustained, human-steerable creative automation.
• Score: 78.0127343695372
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Creative AI is moving from single-step asset generation toward long-horizon multimodal production. Although recent generative models can synthesize high-quality images, videos, audio clips, UI elements, storyboards, slides, and other creative assets, real-world creative work requires more than isolated prompt-output interactions. It involves references, drafts, alternatives, edits, failed attempts, version relations, tool actions, evaluation signals, and human feedback, which together form an evolving project state. Existing prompt-based, chat-based, and node-based generation systems only partially support this state, as they often discard intermediate context, rely on linear conversations, or require manually specified workflows. Recent commercial systems indicate a shift toward agent-assisted creative production, but their closed architectures make it difficult to study how agents represent context, choose tools, revise artifacts, recover from failures, and maintain consistency over time. To address this gap, we introduce JarvisHub, a canvas-native creative agent harness for long-horizon multimodal creation. JarvisHub treats an editable canvas as the user workspace, the agent's external memory, action space, and shared project state, representing multimodal artifacts, dependencies, versions, and feedback as typed canvas nodes and links. Through a three-layer architecture of canvas state, protocol bridge, and agent runtime, JarvisHub enables agents to act within an inspectable and editable creative state. This design moves creative agents beyond isolated tool use toward sustained, human-steerable creative automation, where agents can progressively plan, generate, revise, and organize multimodal projects while users remain able to inspect, guide, and intervene throughout the process.

Related papers

• FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Production [55.71860113185132]
  We introduce FRAMEWORKERS, a task-centric and workspace-grounded multi-agent framework for open-ended video production.<n>A central Director formulates video creation as dynamic task management, continuously editing a Task Stack to determine which subtask to execute next and which sub-agent to invoke.<n>An Assistant serves as the execution layer, grounding each selected task in a shared Workspace, retrieving the required assets and context, invoking the assigned sub-agent, and persisting the resulting artifacts.<n>Experiments show that FRAMEWORKERS outperforms strong planners in routing accuracy, recovers reliably from runtime failures, generalizes to unseen sub
  arXiv  Detail & Related papers  (2026-08-30T14:28:50Z)
• CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration [11.992882604393003]
  We introduce CanvasCraft, a large-scale multimodal tool-use dataset for complex image creation and editing.<n>We also present CanvasAgent, a toolaugmented multimodal agent that learns to orchestrate heterogeneous visual tools through multi-turn interaction.
  arXiv  Detail & Related papers  (2026-07-06T04:57:18Z)
• CutVerse: A Compositional GUI Agents Benchmark for Media Post-Production Editing [30.62336565964961]
  Cutverse is a benchmark designed to evaluate autonomous GUI agents in realistic media post-production environments.<n>We curate expert demonstrations across 7 professional applications (e.g., Premiere Pro, Photoshop) covering complex, long-horizon tasks grounded in authentic editing.
  arXiv  Detail & Related papers  (2026-05-19T07:35:22Z)
• A Versatile Multimodal Agent for Multimedia Content Generation [66.86040734610073]
  We propose a MultiMedia-Agent designed to automate complex content creation tasks.<n>Our agent system includes a data generation pipeline, a tool library for content creation, and a set of metrics for evaluating preference alignment.
  arXiv  Detail & Related papers  (2026-01-06T18:49:47Z)
• Doc To The Future: Infomorphs for Interactive, Multimodal Document Transformation and Generation [4.665470854146965]
  We introduce the notion of "infomorphs" which are modular, user-steerable, AI-augmented transformations that support controlled synthesis, and restructuring of information across formats and modalities.<n>We propose a design space that leverage infomorph-driven document creation by combining Generative AI techniques with user intent and desired information context.<n>DocuCraft allows users to chain together infomorphs that perform operations such as page extraction, content summarization, reformatting, and generation, leveraging Generative AI at each stage to support rich, cross-document and cross-modal transformations.
  arXiv  Detail & Related papers  (2025-12-14T16:25:19Z)
• UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalist [107.04196084992907]
  We introduce UniVA, an omni-capable multi-agent framework for next-generation video generalists.<n>UniVA employs a Plan-and-Act dual-agent architecture that drives a highly automated and proactive workflow.<n>We also introduce UniVA-Bench, a benchmark suite of multi-step video tasks spanning understanding, editing, segmentation, and generation.
  arXiv  Detail & Related papers  (2025-11-11T17:58:13Z)
• Hollywood Town: Long-Video Generation via Cross-Modal Multi-Agent Orchestration [73.65102758687289]
  This study introduces three innovations to improve multi-agent collaboration.<n>First, we propose OmniAgent, a hierarchical, graph-based multi-agent framework for long video generation.<n>Second, inspired by context engineering, we propose hypergraph nodes that enable temporary group discussions.
  arXiv  Detail & Related papers  (2025-10-25T20:34:18Z)
• ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents [40.697759330690815]
  ScreenCoder is a modular multi-agent framework that decomposes the task into three interpretable stages: grounding, planning, and generation.<n>By assigning these distinct responsibilities to specialized agents, our framework achieves significantly higher robustness and fidelity than end-to-end approaches.<n>Our approach achieves state-of-the-art performance in layout accuracy, structural coherence, and code correctness.
  arXiv  Detail & Related papers  (2025-07-30T16:41:21Z)
• CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic Design Generation [13.354283356097563]
  CreatiPoster is a framework that generates editable, multilayer compositions from optional natural-language instructions or assets.<n>To further research, we release a copyright-free corpus of 100,000 multi-layer designs.
  arXiv  Detail & Related papers  (2025-06-12T16:54:39Z)
• OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimization [66.22117723598872]
  We introduce an open-source framework designed to facilitate the development of multimodal web agent. We first train the base model with imitation learning to gain the basic abilities. We then let the agent explore the open web and collect feedback on its trajectories.
  arXiv  Detail & Related papers  (2024-10-25T15:01:27Z)
• Generative Agents: Interactive Simulacra of Human Behavior [86.1026716646289]
  We introduce generative agents--computational software agents that simulate believable human behavior. We describe an architecture that extends a large language model to store a complete record of the agent's experiences. We instantiate generative agents to populate an interactive sandbox environment inspired by The Sims.
  arXiv  Detail & Related papers  (2023-04-07T01:55:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.