SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generation

• URL: http://arxiv.org/abs/2604.19907v1
• Date: Tue, 21 Apr 2026 18:33:15 GMT
• Title: SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generation
• Abstract summary: We propose SceneOrchestra, a trainable orchestration framework for 3D scene creation.<n> SceneOrchestra consists of an orchestrator and a discriminator, which we fine-tune with a two-phase training strategy.<n>Our method achieves state-of-the-art scene quality while reducing runtime compared to previous work.
• Score: 13.62882510697547
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent agentic frameworks for 3D scene synthesis have advanced realism and diversity by integrating heterogeneous generation and editing tools. These tools are organized into workflows orchestrated by an off-the-shelf LLM. Current approaches typically adopt an execute-review-reflect loop: at each step, the orchestrator executes a tool, renders intermediate results for review, and then decides on the tool and its parameters for the next step. However, this design has two key limitations. First, next-step tool selection and parameter configuration are driven by heuristic rules, which can lead to suboptimal execution flows, unnecessary tool invocations, degraded output quality, and increased runtime. Second, rendering and reviewing intermediate results after each step introduces additional latency. To address these issues, we propose SceneOrchestra, a trainable orchestration framework that optimizes the tool-call execution flow and eliminates the step-by-step review loop, improving both efficiency and output quality. SceneOrchestra consists of an orchestrator and a discriminator, which we fine-tune with a two-phase training strategy. In the first phase, the orchestrator learns context-aware tool selection and complete tool-call trajectory generation, while the discriminator is trained to assess the quality of full trajectories, enabling it to select the best trajectory from multiple candidates. In the second phase, we perform interleaved training, where the discriminator adapts to the orchestrator's evolving trajectory distribution and distills its discriminative capability back into the orchestrator. At inference, we only use the orchestrator to generate and execute full tool-call trajectories from instructions, without requiring the discriminator. Extensive experiments show that our method achieves state-of-the-art scene quality while reducing runtime compared to previous work.

Related papers

• DataOrchestra: Learning to Orchestrate Per-Example Curation of Pretraining Data [27.78341478407496]
  We propose DataOrchestra, a framework that unifies different processing operations and orchestrates an example-specific pipeline for each example.<n>We pretrain models from 0.5B to 7B from scratch on web data processed by DataOrchestra and observe stable average gains over individual data-processing methods across 11 benchmarks.
  arXiv  Detail & Related papers  (2026-07-27T17:54:12Z)
• DaV-Gen: End-to-End Generative Retrieval via Draft-and-Verify [4.546259713278263]
  textbfDaV-Gen is a novel unified solution designed to fundamentally the paradigm for both search and recommendation.<n>During training, the model is concurrently optimized for both candidate drafting and fine-grained verification.<n>This is achieved by a composite loss function that jointly trains the model on two distinct but related objectives.
  arXiv  Detail & Related papers  (2026-07-09T11:22:50Z)
• Small Model as Master Orchestrator: Learning Unified Agent-Tool Orchestration with Parallel Subtask Decomposition [61.291733522717415]
  Agent-as-Tool is a unified parallel orchestration paradigm that mitigates both agents and tools into a standardized, learnable action space.<n>ParaManager decouples planning decisions from subtask solving, enabling state-aware parallel subtask decomposition, delegation, and asynchronous execution.<n>Experiments show that ParaManager achieves strong performance across multiple benchmarks and exhibits robust generalization under unseen model pools.
  arXiv  Detail & Related papers  (2026-04-18T14:41:27Z)
• Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards [76.49428173793386]
  LLMs must invoke multiple dependent APIs in the correct order while propagating intermediate outputs.<n>Training models to handle such faces two obstacles: existing environments focus on simple per-turn function calls with simulated data, and binary rewards.<n>We present a framework addressing both challenges. First, we construct a reinforcement learning environment backed by a large-scale cache of real API responses, enabling a data synthesis pipeline that samples valid multi-step orchestration traces.<n>Second, we propose a graduated reward design that decomposes correctness into atomic validity and orchestration.
  arXiv  Detail & Related papers  (2026-03-25T18:31:39Z)
• SkillOrchestra: Learning to Route Agents via Skill Transfer [65.50924963973286]
  We introduce SkillOrchestra, a framework for skill-aware orchestration.<n>SkillOrchestra learns fine-grained skills from execution experience and models agent-specific competence and cost under those skills.<n>At deployment, the orchestrator infers the skill demands of the current interaction and selects agents that best satisfy them under an explicit performance-cost trade-off.
  arXiv  Detail & Related papers  (2026-02-23T10:17:25Z)
• Robust and Efficient Tool Orchestration via Layered Execution Structures with Reflective Correction [55.13278005189741]
  We model tool orchestration as learning a layered execution structure that captures high-level tool dependencies.<n>We introduce a schema-aware reflective correction mechanism that detects and repairs errors locally.<n>This design confines errors to individual tool calls and avoids re-planning entire execution trajectories.
  arXiv  Detail & Related papers  (2026-02-21T22:20:01Z)
• AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration [36.04766562662082]
  AOrchestra is a framework-agnostic agent abstraction that models any agent as a Instruction, Context Tools, Model.<n>It curates task-relevant context, tools and models, and delegates execution via on-the-fly automatic agent creation.<n>AOrchestra achieves 16.28% relative improvement against the strongest baseline when paired with Gemini.
  arXiv  Detail & Related papers  (2026-02-03T17:46:16Z)
• Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents [52.30603055218294]
  Trajectory2Task is a verifiable data generation pipeline for studying tool use at scale under three realistic user scenarios.<n>It converts valid tool-call trajectories into user-facing tasks with controlled intent adaptations.<n>We benchmark seven state-of-the-art LLMs on the generated complex user scenario tasks and observe frequent failures.
  arXiv  Detail & Related papers  (2026-01-28T00:36:13Z)
• ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration [110.24410841004777]
  We show that small orchestrators managing other models and a variety of tools can both push the upper bound of intelligence.<n>We introduce ToolOrchestra, a method for training small orchestrators that coordinate intelligent tools.<n>Using ToolOrchestra, we produce Orchestrator, an 8B model that achieves higher accuracy at lower cost than previous tool-use agents.
  arXiv  Detail & Related papers  (2025-11-26T18:59:46Z)
• SOCIA-Nabla: Textual Gradient Meets Multi-Agent Orchestration for Automated Simulator Generation [16.707429579441435]
  SOCIA-Nabla is an end-to-end, agentic framework that treats simulator construction asinstance optimization over code within a textual graph.<n>Specialized computation-driven agents are embedded as graph nodes, and a workflow manager executes a loss-driven loop: code synthesis -> execution -> evaluation -> code repair.
  arXiv  Detail & Related papers  (2025-10-21T12:00:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.