CoLF: Learning Consistent Leader-Follower Policies for Vision-Language-Guided Multi-Robot Cooperative Transport

• URL: http://arxiv.org/abs/2602.07776v1
• Date: Sun, 08 Feb 2026 02:21:47 GMT
• Title: CoLF: Learning Consistent Leader-Follower Policies for Vision-Language-Guided Multi-Robot Cooperative Transport
• Abstract summary: We address vision-language-guided multi-robot cooperative transport, where each robot grounds natural-language instructions from onboard camera observations.<n>A key challenge in this decentralized setting is perceptual misalignment across robots, where viewpoint differences and language ambiguity can yield inconsistent interpretations.<n>We propose Consistent Leader-Follower (CoLF), a multi-agent reinforcement learning framework for stable leader-follower role differentiation.
• Score: 4.619525549930793
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this study, we address vision-language-guided multi-robot cooperative transport, where each robot grounds natural-language instructions from onboard camera observations. A key challenge in this decentralized setting is perceptual misalignment across robots, where viewpoint differences and language ambiguity can yield inconsistent interpretations and degrade cooperative transport. To mitigate this problem, we adopt a dependent leader-follower design, where one robot serves as the leader and the other as the follower. Although such a leader-follower structure appears straightforward, learning with independent and symmetric agents often yields symmetric or unstable behaviors without explicit inductive biases. To address this challenge, we propose Consistent Leader-Follower (CoLF), a multi-agent reinforcement learning (MARL) framework for stable leader-follower role differentiation. CoLF consists of two key components: (1) an asymmetric policy design that induces leader-follower role differentiation, and (2) a mutual-information-based training objective that maximizes a variational lower bound, encouraging the follower to predict the leader's action from its local observation. The leader and follower policies are jointly optimized under the centralized training and decentralized execution (CTDE) framework to balance task execution and consistent cooperative behaviors. We validate CoLF in both simulation and real-robot experiments using two quadruped robots. The demonstration video is available at https://sites.google.com/view/colf/.

Related papers

• MobileVLA-R1 2.0: RL-Enhanced Reasoning for Mobile Robot Control [63.85634404397772]
  MobileVLA-R1 2.0 is an RL-enhanced vision-language-action framework for mobile robots.<n>It learns multi-granularity reasoning over embodied trajectories through supervised Chain-of-Thought (CoT) alignment and reinforcement learning.<n>We conduct evaluations on language-guided navigation, quadruped control, and humanoid mobile manipulation, covering VLN-CE, QUARD, and real-world deployments on Unitree Go2 and G1 robots.
  arXiv  Detail & Related papers  (2026-09-05T20:31:14Z)
• Bidirectional Tutoring for Developmental Motor Learning in Robots: Co-Developed Interaction Dynamics Support Stable Learning [1.1372912154713897]
  Motor-skill learning in robots is often treated as a unidirectional process in which robots passively receive demonstrations from tutors.<n>We implement the developmental learning framework using a free-energy-principle-based neural network extended with generative replay.
  arXiv  Detail & Related papers  (2026-06-18T02:51:29Z)
• HumanoidArena: Benchmarking Egocentric Hierarchical Whole-body Learning [38.43380992093738]
  We introduce HumanoidArena, a simulation-first benchmark for egocentric hierarchical whole-body learning.<n>A high-level policy converts egocentric vision, proprioception, and instructions into a compact whole-body action, which is subsequently executed by a low-level GMT.<n>Experiments show that hierarchical control enables learned policies to solve diverse leg-critical interactions, but performance is strongly tracker-conditioned and cross-GMT transfer remains fragile.
  arXiv  Detail & Related papers  (2026-06-16T12:01:31Z)
• CHORUS: Decentralized Multi-Embodiment Collaboration with One VLA Policy [67.76658561040625]
  CHORUS is a framework that adapts a single vision-language-action (VLA) backbone to control diverse, multi-robot teams.<n>In experiments, CHORUS achieves a 64% point improvement over decentralized, from-scratch models.<n>Results show that a shared VLA backbone is capable of achieving decentralized multi-robot collaboration.
  arXiv  Detail & Related papers  (2026-06-10T17:26:08Z)
• VAIC: Vision-Guided Humanoid Agile Object Interaction Control via Decoupled Commands [56.029505206531155]
  Vision Guided Agile Interaction Control (VAIC) is a unified framework that bridges the gap between current controllers and humanoid robots.<n>First, a privileged teacher policy masters diverse interaction skills using precise object kinematics and exact environmental states.<n>Second, a deployable student policy distills these capabilities by replacing full body tracking with velocity targets across multiple axes.
  arXiv  Detail & Related papers  (2026-06-08T09:52:55Z)
• Seeing Together: Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models [76.29781698910202]
  We propose a framework for fine-grained cooperative spatial reasoning.<n> SP-CoR combines dynamics-aware multi-robot frame sampling, spectral- and physics-guided view fusion, and physics-aligned prompt distillation.<n>It consistently improves cooperative reasoning, outperforming the strongest finetuned baseline by +3.87% on Habitat and +7.12% on iGibson.
  arXiv  Detail & Related papers  (2026-05-18T14:04:26Z)
• Cognition to Control - Multi-Agent Learning for Human-Humanoid Collaborative Transport [36.304887250687536]
  This paper introduces cognition-to-control (C2C), a three-layer hierarchy that makes the deliberation-to-control pathway explicit.<n> Experiments on collaborative manipulation tasks show higher success and robustness than single-agent and end-to-end baselines.
  arXiv  Detail & Related papers  (2026-03-04T06:24:55Z)
• Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation [50.01551945190676]
  Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. We propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures. We demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation.
  arXiv  Detail & Related papers  (2024-01-22T18:58:22Z)
• Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deployment [88.06408322210025]
  We study the problem of adapting on-the-fly to novel scenarios during deployment.<n>Our approach, RObust Autonomous Modulation (ROAM), introduces a mechanism based on the perceived value of pre-trained behaviors.<n>We demonstrate that ROAM enables a robot to adapt rapidly to changes in dynamics both in simulation and on a real Go1 quadruped.
  arXiv  Detail & Related papers  (2023-11-02T08:22:28Z)
• Multi-Agent Deep Reinforcement Learning for Cooperative and Competitive Autonomous Vehicles using AutoDRIVE Ecosystem [1.1893676124374688]
  We introduce AutoDRIVE Ecosystem as an enabler to develop physically accurate and graphically realistic digital twins of Nigel and F1TENTH. We first investigate an intersection problem using a set of cooperative vehicles (Nigel) that share limited state information with each other in single as well as multi-agent learning settings. We then investigate an adversarial head-to-head autonomous racing problem using a different set of vehicles (F1TENTH) in a multi-agent learning setting using an individual policy approach.
  arXiv  Detail & Related papers  (2023-09-18T02:43:59Z)
• Learning Vision-based Pursuit-Evasion Robot Policies [54.52536214251999]
  We develop a fully-observable robot policy that generates supervision for a partially-observable one. We deploy our policy on a physical quadruped robot with an RGB-D camera on pursuit-evasion interactions in the wild.
  arXiv  Detail & Related papers  (2023-08-30T17:59:05Z)
• Stackelberg Games for Learning Emergent Behaviors During Competitive Autocurricula [35.88217121803472]
  This paper proposes a novel game-theoretic algorithm, Stackelberg Multi-Agent Deep Deterministic Policy Gradient (ST-MADDPG) It formulates a two-player MARL problem as a Stackelberg game with one player as the leader' and the other as the follower' in a hierarchical interaction structure wherein the leader has an advantage. By exploiting the leader's advantage, ST-MADDPG improves the quality of a co-evolution process and results in more sophisticated and complex strategies that work well even against an unseen strong opponent.
  arXiv  Detail & Related papers  (2023-05-04T19:27:35Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)
• "No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy [70.45420918526926]
  We present LILAC, a framework for incorporating and adapting to natural language corrections online during execution. Instead of discrete turn-taking between a human and robot, LILAC splits agency between the human and robot. We show that our corrections-aware approach obtains higher task completion rates, and is subjectively preferred by users.
  arXiv  Detail & Related papers  (2023-01-06T15:03:27Z)
• Distributed Differentiable Dynamic Game for Multi-robot Coordination [5.835070654168429]
  We formulate multi-robot coordination as a dynamic game, where the behavior of a robot is dictated by its own dynamics and objective that also depends on others' behavior. This paper develops a Distributed Differentiable Dynamic Game (D3G) framework, which can efficiently solve the forward and inverse problems in multi-robot coordination.
  arXiv  Detail & Related papers  (2022-07-18T19:06:18Z)
• Interactive Dynamic Walking: Learning Gait Switching Policies with Generalization Guarantees [8.554761233491236]
  We consider the problem of adapting a walking bipedal robot to follow a leading co-worker while engaging in tasks that require physical interaction. Our approach relies on switching among a family of Dynamic Movement Primitives as governed by a supervisor. We demonstrate the efficacy of our approach by training a neural-network supervisor to adapt the gait of a dynamically walking biped to a leading collaborator whose intended trajectory is not known explicitly.
  arXiv  Detail & Related papers  (2021-09-28T01:09:13Z)
• Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations [52.696205074092006]
  Generalization Through Imitation (GTI) is a two-stage offline imitation learning algorithm. GTI exploits a structure where demonstrated trajectories for different tasks intersect at common regions of the state space. In the first stage of GTI, we train a policy that leverages intersections to have the capacity to compose behaviors from different demonstration trajectories together. In the second stage of GTI, we train a goal-directed agent to generalize to novel start and goal configurations.
  arXiv  Detail & Related papers  (2020-03-13T02:25:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.