One Supervisor, Many Modalities: Adaptive Tool Orchestration for Autonomous Queries

• URL: http://arxiv.org/abs/2603.11545v2
• Date: Fri, 13 Mar 2026 07:38:03 GMT
• Title: One Supervisor, Many Modalities: Adaptive Tool Orchestration for Autonomous Queries
• Abstract summary: We present an agentic AI framework for autonomous multimodal query processing that coordinates specialized tools across text, image, audio, video, and document modalities.<n>A central Supervisor dynamically decomposes user queries, delegates subtasks to modality-appropriate tools.<n>For text-only queries, the framework uses learned routing via RouteLLM, while non-text paths use SLM-assisted modality decomposition.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present an agentic AI framework for autonomous multimodal query processing that coordinates specialized tools across text, image, audio, video, and document modalities. A central Supervisor dynamically decomposes user queries, delegates subtasks to modality-appropriate tools (e.g., object detection, OCR, speech transcription), and synthesizes results through adaptive routing strategies rather than predetermined decision trees. For text-only queries, the framework uses learned routing via RouteLLM, while non-text paths use SLM-assisted modality decomposition. Evaluated on 2,847 queries across 15 task categories, our framework achieves 72% reduction in time-to-accurate-answer, 85% reduction in conversational rework, and 67% cost reduction compared to the matched hierarchical baseline while maintaining accuracy parity. These results demonstrate that intelligent centralized orchestration fundamentally improves multimodal AI deployment economics.

Related papers

• Knowledge Reutilization in Meta-Reinforcement Learning [51.07319647819108]
  We propose a meta-knowledge reutilization framework that learns task-level knowledge on a dynamics-simplified agent and transfers it to heterogeneous agents.<n>We show that our framework reduces final-step tracking error by 94.75% -- 99.79% compared with recent state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-06-16T16:32:28Z)
• Do Composed Image Retrieval Benchmarks Require Multimodal Composition? [86.99911649534795]
  We show that strong performance on Composed Image Retrieval benchmarks is assumed to require multimodal composition.<n>Across four widely used CIR benchmarks and eleven Generalist Multimodal Embedding models, a large fraction of queries can be solved using a single modality.<n>High CIR performance can arise from unimodal signals rather than true multimodal composition.
  arXiv  Detail & Related papers  (2026-05-14T12:56:36Z)
• Adaptive Chunking: Optimizing Chunking-Method Selection for RAG [0.0]
  We introduce Adaptive Chunking, a framework that selects the most suitable chunking strategy for each document.<n>On a diverse corpus spanning legal, technical, and social science domains, our metric-merge adaptive method significantly improves downstream RAG performance.
  arXiv  Detail & Related papers  (2026-03-26T11:20:52Z)
• Task-Aware LoRA Adapter Composition via Similarity Retrieval in Vector Databases [3.4869850730657728]
  We present a novel framework for dynamic LoRA adapter composition that leverages similarity retrieval in vector databases.<n>Our approach constructs a task-aware vector database by embedding training examples from 22 spanning commonsense reasoning, question answering, natural language inference, and sentiment analysis.<n>Our framework requires no additional retriever training, operates with frozen embeddings, and enables efficient, interpretable adapter composition.
  arXiv  Detail & Related papers  (2026-02-01T22:20:04Z)
• Efficient Multimodal Planning Agent for Visual Question-Answering [67.26245301307539]
  This paper proposes a method that trains a multimodal planning agent, dynamically decomposing the mRAG pipeline to solve the VQA task.<n>In our experiments, the agent can help reduce redundant computations, cutting search time by over 60% compared to existing methods.
  arXiv  Detail & Related papers  (2026-01-28T14:58:59Z)
• FUSE : Failure-aware Usage of Subagent Evidence for MultiModal Search and Recommendation [1.3618516245643617]
  Multimodal creative assistants decompose user goals and route tasks to subagents for layout, styling, retrieval, and generation.<n>FUSE implements seven context budgeting strategies: context compression, chain-of-thought reasoning, mini-shot optimization, retrieval-augmented context, two-stage processing, and zero-shot minimalism.
  arXiv  Detail & Related papers  (2025-11-15T07:55:51Z)
• ContextNav: Towards Agentic Multimodal In-Context Learning [85.05420047017513]
  ContextNav is an agentic framework that integrates the scalability of automated retrieval with the quality and adaptiveness of human-like curation.<n>It builds a resource-aware multimodal embedding pipeline, maintains a retrievable vector database, and applies agentic retrieval and structural alignment to construct noise-resilient contexts.<n> Experimental results demonstrate that ContextNav achieves state-of-the-art performance across various datasets.
  arXiv  Detail & Related papers  (2025-10-06T07:49:52Z)
• How Can Input Reformulation Improve Tool Usage Accuracy in a Complex Dynamic Environment? A Study on $τ$-bench [58.114899897566964]
  In a multi-turn conversational environment, large language models (LLMs) often struggle with consistent reasoning and adherence to domain-specific policies.<n>We propose the Input-Reformulation Multi-Agent (IRMA) framework, which automatically reformulates user queries augmented with relevant domain rules.<n>IRMA significantly outperforms ReAct, Function Calling, and Self-Reflection by 16.1%, 12.7%, and 19.1%, respectively.
  arXiv  Detail & Related papers  (2025-08-28T15:57:33Z)
• Planning Agents on an Ego-Trip: Leveraging Hybrid Ego-Graph Ensembles for Improved Tool Retrieval in Enterprise Task Planning [0.0]
  We propose a Knowledge Graph-based tool retrieval framework that captures the semantic relationships between tools and their functional dependencies.<n>Our retrieval algorithm leverages ensembles of 1-hop ego tool graphs to model direct and indirect connections between tools.<n>Results demonstrate that our tool graph-based method achieves 91.85% tool coverage on the micro-average Complete Recall metric.
  arXiv  Detail & Related papers  (2025-08-07T22:41:12Z)
• RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory [57.449129198822476]
  RCR is a role-aware context routing framework for multi-agent large language model (LLM) systems.<n>It dynamically selects semantically relevant memory subsets for each agent based on its role and task stage.<n>A lightweight scoring policy guides memory selection, and agent outputs are integrated into a shared memory store.
  arXiv  Detail & Related papers  (2025-08-06T21:59:34Z)
• Compliance Brain Assistant: Conversational Agentic AI for Assisting Compliance Tasks in Enterprise Environments [2.8724171056550256]
  Compliance Brain Assistant (CBA) is a conversational, agentic AI assistant designed to boost the efficiency of daily compliance tasks for personnel in enterprise environments.<n>To strike a good balance between response quality and latency, we design a user query router that can intelligently choose between FastTrack mode and FullAgentic mode.
  arXiv  Detail & Related papers  (2025-07-23T07:51:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.