HECTOR: Human-centric Hierarchical Coordination and Supervision of Robotic Fleets under Continual Temporal Tasks

• URL: http://arxiv.org/abs/2604.10892v1
• Date: Mon, 13 Apr 2026 01:39:19 GMT
• Title: HECTOR: Human-centric Hierarchical Coordination and Supervision of Robotic Fleets under Continual Temporal Tasks
• Abstract summary: This work proposes a human-centric coordination and supervision scheme (HECTOR) for large-scale robotic fleets under continual and uncertain temporal tasks.<n>It consists of three hierarchical layers: (I) the bidirectional and multimodal protocol of online human-fleet interaction, where the operator interacts with and supervises the whole fleet; (II) the rolling assignment of currently-known tasks to teams within a certain horizon, and (III) the dynamic coordination within a team given the detected subtasks during online execution.
• Score: 7.155003467581939
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robotic fleets can be extremely efficient when working concurrently and collaboratively, e.g., for delivery, surveillance, search and rescue. However, it can be demanding or even impractical for an operator to directly control each robot. Thus, autonomy of the fleet and its online interaction with the operator are both essential, particularly in dynamic and partially unknown environments. The operator might need to add new tasks, cancel some tasks, change priorities and modify planning results. How to design the procedure for these interactions and efficient algorithms to fulfill these needs have been mostly neglected in the related literature. Thus, this work proposes a human-centric coordination and supervision scheme (HECTOR) for large-scale robotic fleets under continual and uncertain temporal tasks. It consists of three hierarchical layers: (I) the bidirectional and multimodal protocol of online human-fleet interaction, where the operator interacts with and supervises the whole fleet; (II) the rolling assignment of currently-known tasks to teams within a certain horizon, and (III) the dynamic coordination within a team given the detected subtasks during online execution. The overall mission can be as general as temporal logic formulas over collaborative actions. Such hierarchical structure allows human interaction and supervision at different granularities and triggering conditions, to both improve computational efficiency and reduce human effort. Extensive human-in-the-loop simulations are performed over heterogeneous fleets under various temporal tasks and environmental uncertainties.

Related papers

• HRIBench: Benchmarking Interaction-Centric Human-Robot Collaboration [62.19372189311434]
  We introduce HRIBench, a diagnostic benchmark for intent-aware human-robot collaboration based on executable scenarios.<n>HRIBench represents collaborative tasks as structured scenario scripts that explicitly model agent roles, temporal dependencies, coordination constraints, and human behavior distributions.<n>Results show that current foundation robot policies struggle substantially in collaborative settings despite strong manipulation ability.
  arXiv  Detail & Related papers  (2026-07-05T09:15:35Z)
• VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands [56.029505206531155]
  Vision Guided Agile Interaction Control (VAIC) is a unified framework that bridges the gap between current controllers and humanoid robots.<n>First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states.<n>Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes.
  arXiv  Detail & Related papers  (2026-06-08T09:52:55Z)
• Higher Order Reasoning for Collaborative Communicationless Mobile Robot Operations [4.239775815863115]
  In communicationless environments, multi-robot systems must operate without the constant information exchange that many coordination strategies typically assume.<n>This paper presents a novel dynamic epistemic planning framework that enables implicit coordination and long horizon planning through higher-order reasoning among robots.
  arXiv  Detail & Related papers  (2026-05-21T02:22:09Z)
• HULK: Large-scale Hierarchical Coordination under Continual and Uncertain Temporal Tasks [5.337512368549053]
  This work tackles the large-scale coordination problem under continual and uncertain temporal tasks.<n>The proposed hierarchical framework, HULK, consists of two interleaved layers: the rolling assignment of currently known tasks to subteams within a certain horizon, and the dynamic coordination within a subteam given the detected subtasks during online execution.<n>The method is validated rigorously over large-scale heterogeneous systems under various temporal tasks and environment uncertainties.
  arXiv  Detail & Related papers  (2026-05-09T06:10:40Z)
• Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs [52.26652574704317]
  Large language models (LLMs) are increasingly deployed in teams, yet existing coordination approaches often occupy two extremes.<n>We introduce Language Agent Teams for Task Evolution (LATTE), a framework for coordinating LLM teams inspired by distributed systems.
  arXiv  Detail & Related papers  (2026-05-07T14:19:17Z)
• DeCoNav: Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation [51.978605314711835]
  Long-horizon collaborative vision-language navigation (VLN) is critical for multi-robot systems to accomplish complex tasks beyond the capability of a single agent.<n>We present Dialog enhanced Long-Horizon Collaborative Vision-Language Navigation (DeCoNav)<n>DeCoNav is a decentralized framework that couples event-triggered dialogue with dynamic task allocation and replanning for real-time, adaptive coordination.
  arXiv  Detail & Related papers  (2026-04-14T09:11:55Z)
• Federated Single-Agent Robotics: Multi-Robot Coordination Without Intra-Robot Multi-Agent Fragmentation [14.412476605788482]
  We present Federated Single-Agent Robotics, a runtime architecture for multi-robot coordination built on single-agent robot runtimes.<n>We formalize key coordination relations including authority delegation, inter-robot capability requests, local-versus-fleet recovery boundaries, and hierarchical human supervision.<n>Results show statistically significant gains in governance locality and recovery containment against decomposition-heavy baselines.
  arXiv  Detail & Related papers  (2026-04-13T05:51:13Z)
• MOSAIC: Modular Scalable Autonomy for Intelligent Coordination of Heterogeneous Robotic Teams [10.664029115378813]
  We present a scalable autonomy framework for multi-robot scientific exploration using a unified mission abstraction.<n>The proposed framework enables robust, scalable multi-robot scientific exploration with limited operator intervention.
  arXiv  Detail & Related papers  (2026-01-30T14:46:15Z)
• Heterogeneous Robot Collaboration in Unstructured Environments with Grounded Generative Intelligence [54.91177026001217]
  Large language model (LLM)-enabled teaming methods typically assume well-structured and known environments.<n>We present SPINE-HT, a framework that addresses these limitations by grounding the reasoning abilities of LLMs in the context of a heterogeneous robot team.<n>Our framework achieves nearly twice the success rate compared to prior LLM-enabled heterogeneous teaming approaches.
  arXiv  Detail & Related papers  (2025-10-30T18:24:38Z)
• HHI-Assist: A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario [63.77482302352545]
  HHI-Assist is a dataset comprising motion capture clips of human-human interactions in assistive tasks.<n>Our work has the potential to significantly enhance robotic assistance policies.
  arXiv  Detail & Related papers  (2025-09-12T09:38:17Z)
• COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models [49.24666980374751]
  COHERENT is a novel LLM-based task planning framework for collaboration of heterogeneous multi-robot systems.<n>A Proposal-Execution-Feedback-Adjustment mechanism is designed to decompose and assign actions for individual robots.<n>The experimental results show that our work surpasses the previous methods by a large margin in terms of success rate and execution efficiency.
  arXiv  Detail & Related papers  (2024-09-23T15:53:41Z)
• Large Language Models for Orchestrating Bimanual Robots [19.60907949776435]
  We present LAnguage-model-based Bimanual ORchestration (LABOR) to analyze task configurations and devise coordination control policies. We evaluate our method through simulated experiments involving two classes of long-horizon tasks using the NICOL humanoid robot.
  arXiv  Detail & Related papers  (2024-04-02T15:08:35Z)
• Simulation of robot swarms for learning communication-aware coordination [0.0]
  We train end-to-end Neural Networks that take as input local observations obtained from an omniscient centralised controller. Experiments are run in Enki, a high-performance open-source simulator for planar robots.
  arXiv  Detail & Related papers  (2023-02-25T17:17:40Z)
• AdverSAR: Adversarial Search and Rescue via Multi-Agent Reinforcement Learning [4.843554492319537]
  We propose an algorithm that allows robots to efficiently coordinate their strategies in the presence of adversarial inter-agent communications. It is assumed that the robots have no prior knowledge of the target locations, and they can interact with only a subset of neighboring robots at any time. The effectiveness of our approach is demonstrated on a collection of prototype grid-world environments.
  arXiv  Detail & Related papers  (2022-12-20T08:13:29Z)
• ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulation [99.2543521972137]
  ReLMoGen is a framework that combines a learned policy to predict subgoals and a motion generator to plan and execute the motion needed to reach these subgoals. Our method is benchmarked on a diverse set of seven robotics tasks in photo-realistic simulation environments. ReLMoGen shows outstanding transferability between different motion generators at test time, indicating a great potential to transfer to real robots.
  arXiv  Detail & Related papers  (2020-08-18T08:05:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.