Open-ended Multi-agent Autocurricula via Visual Inspection of Policies with Multi-modal LLMs

• URL: http://arxiv.org/abs/2607.08193v1
• Date: Thu, 09 Jul 2026 07:48:36 GMT
• Title: Open-ended Multi-agent Autocurricula via Visual Inspection of Policies with Multi-modal LLMs
• Abstract summary: Open-ended curricula in Reinforcement Learning (RL) aim to train generally-capable agents by identifying tasks that facilitate learning increasingly complex skills.<n>Previous work has explored using scalar task scores or textual summaries of the agent's behavior.<n>Here we study a different approach: directly inspecting policy behavior via recorded episode videos.
• Score: 2.5234156040689233
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open-ended curricula in Reinforcement Learning (RL) aim to train generally-capable agents by identifying tasks that facilitate learning increasingly complex skills. A major challenge when designing such curricula is assessing task difficulty relative to the agent's current learning progress. While previous work has explored using scalar task scores or textual summaries of the agent's behavior, here we study a different approach: directly inspecting policy behavior via recorded episode videos. We introduce a simple yet effective instantiation of this approach which leverages a Video Language Model (VLM) to both process these videos and provide curriculum recommendations, which we call Visual Inspection of Policies (VIP). Since videos can naturally contain any number of controllable agents, we empirically study VIP on the StarCraft Multi-Agent Challenge (SMAC). We show that even with a lightweight and openly accessible VLM (VideoLLaMa2-7B), VIP can use policy videos to generate more effective curricula than both its text-only ablation and methods that rely on scalar task scores.

Related papers

• Single-agent vs. Multi-agents for Automated Video Analysis of On-Screen Collaborative Learning Behaviors [0.0]
  On-screen learning behavior provides valuable insights into how students seek, use, and create information during learning.<n>Recent development of Vision Language Models (VLMs) offers new opportunities to automate the labor-intensive manual coding.<n>This study demonstrates the effectiveness of VLM-based Multi-agent System for video analysis and contributes a scalable framework for multimodal data analytics.
  arXiv  Detail & Related papers  (2026-04-04T08:01:02Z)
• Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations [22.561305437484975]
  We propose Behavior Cloning from Videos via Latent Representations (BCV-LR) to achieve imitation learning from videos.<n>BCV-LR extracts action-related latent features from high-dimensional video inputs through self-supervised tasks.<n>We conduct extensive experiments on a set of challenging visual tasks, including both discrete control and continuous control.
  arXiv  Detail & Related papers  (2025-12-25T09:11:14Z)
• Training Multi-Image Vision Agents via End2End Reinforcement Learning [51.81337984526068]
  We propose IMAgent, an open-source vision agent trained via end-to-end reinforcement learning.<n>By leveraging a multi-agent system, we generate challenging and visually-rich multi-image QA pairs.<n>We develop two specialized tools for visual reflection and confirmation, allowing the model to proactively reallocate its attention to image content.
  arXiv  Detail & Related papers  (2025-12-05T10:02:38Z)
• VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learning [30.278740496355507]
  We propose a novel Multi-agent system for video understanding, namely VideoChat-M1.<n>Instead of using a single or fixed policy, VideoChat-M1 adopts a distinct Collaborative Policy Planning ( CPP) paradigm with multiple policy agents.<n>We show that VideoChat-M1 achieves SOTA performance across eight benchmarks spanning four tasks.
  arXiv  Detail & Related papers  (2025-11-24T07:04:51Z)
• Sample-efficient Unsupervised Policy Cloning from Ensemble Self-supervised Labeled Videos [7.827978803804189]
  Unsupervised Policy from Ensemble Self-supervised labeled Videos (SV) is a novel framework to efficiently learn policies from action-free videos without rewards and any other supervision.<n>SV trains a video labeling model to infer the expert actions in expert videos through several combined self-supervised tasks.<n>After a sample-efficient, unsupervised, and iterative training process, SV obtains an advanced policy based on a robust video labeling model.
  arXiv  Detail & Related papers  (2024-12-14T10:12:22Z)
• TANGO: Training-free Embodied AI Agents for Open-world Tasks [11.029387480118652]
  We propose TANGO, an approach that extends the program composition via LLMs already observed for images.<n>We show how a single model can address diverse tasks without additional training.<n>We evaluate our approach on three key Embodied AI tasks: Open-Set ObjectGoal Navigation, Multi-Modal Lifelong Navigation, and Open Embodied Question Answering.
  arXiv  Detail & Related papers  (2024-12-05T21:52:20Z)
• VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding [65.12464615430036]
  This paper introduces a Video Understanding and Reasoning Framework (VURF) based on the reasoning power of Large Language Models (LLMs)<n>Ours is a novel approach to extend the utility of LLMs in the context of video tasks, leveraging their capacity to generalize from minimal input and output demonstrations within a contextual framework.
  arXiv  Detail & Related papers  (2024-03-21T18:00:00Z)
• Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization [53.510942601223626]
  Large Language Models (LLMs) exhibit robust problem-solving capabilities for diverse tasks. These task solvers necessitate manually crafted prompts to inform task rules and regulate behaviors. We propose Agent-Pro: an LLM-based Agent with Policy-level Reflection and Optimization.
  arXiv  Detail & Related papers  (2024-02-27T15:09:20Z)
• Vision-Language Models Provide Promptable Representations for Reinforcement Learning [67.40524195671479]
  We propose a novel approach that uses the vast amounts of general and indexable world knowledge encoded in vision-language models (VLMs) pre-trained on Internet-scale data for embodied reinforcement learning (RL) We show that our approach can use chain-of-thought prompting to produce representations of common-sense semantic reasoning, improving policy performance in novel scenes by 1.5 times.
  arXiv  Detail & Related papers  (2024-02-05T00:48:56Z)
• DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent) [73.10899129264375]
  This paper explores DoraemonGPT, a comprehensive and conceptually elegant system driven by LLMs to understand dynamic scenes.<n>Given a video with a question/task, DoraemonGPT begins by converting the input video into a symbolic memory that stores task-related attributes.<n>We extensively evaluate DoraemonGPT's effectiveness on three benchmarks and several in-the-wild scenarios.
  arXiv  Detail & Related papers  (2024-01-16T14:33:09Z)
• Adapting Pre-trained Language Models to Vision-Language Tasks via Dynamic Visual Prompting [83.21164539349273]
  Pre-trained language models (PLMs) have played an increasing role in multimedia research. In this paper, we focus on exploring PLMs as a stand-alone model for vision-language reasoning tasks. We propose a novel transfer learning approach for PLMs, termed Dynamic Visual Prompting (DVP)
  arXiv  Detail & Related papers  (2023-06-01T07:19:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.