Iron: Intent-Aligned and Retrospective Dual Learning Framework for Enhancing Generalist Virtual Agents

• URL: http://arxiv.org/abs/2608.27866v1
• Date: Fri, 28 Aug 2026 03:15:11 GMT
• Title: Iron: Intent-Aligned and Retrospective Dual Learning Framework for Enhancing Generalist Virtual Agents
• Abstract summary: We introduce Iron, an intent-aligned framework for training GUI agents.<n>Iron employs a stepwise cycle-consistent reward to achieve fine-grained alignment between low-level actions and high-level intents.<n>Experiments demonstrate that Iron-trained generalist agents consistently improve performance on cross-environment and cross-device tasks.
• Score: 31.811763840603614
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Achieving virtual agents capable of automating tasks across diverse digital environments remains a pivotal challenge in Embodied AI. While Multimodal Large Language Models (MLLMs) offer enhanced visual perception and reasoning, their agentic deployment faces three challenges: costly data annotation, imprecise action-intent alignment, and inefficient exploration from discarded failed trajectories. To address these, we introduce Iron, an intent-aligned, self-improved, and annotation-efficient framework for training GUI agents. Iron employs a novel dual learning strategy that utilizes a stepwise cycle-consistent (SCC) reward to achieve fine-grained alignment between low-level actions and high-level intents, thereby improving instruction grounding and intent understanding. Concurrently, Iron introduces a hindsight reproduction mechanism to repurpose failed trajectories for training, improving both learning efficiency and task diversity. Extensive experiments demonstrate that Iron-trained generalist agents consistently improve performance on cross-environment and cross-device tasks, outperforming models trained with three times more data. Iron also achieves a substantial 25.06% relative improvement on unseen web tasks, with further gains observed on inherently complex tasks, demonstrating the feasibility of building more capable virtual agents.

Related papers

• Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learning [4.847786820784619]
  Inverse reinforcement learning (IRL) provides a powerful framework for learning from demonstrations.<n>We introduce Multitask discriminator Proximity-Guided IRL (MPG)<n>MPG learns two complementary reward components: (1) a generalizable discriminator that transfers shared structure across related tasks to identify expert behavior in a new task, and (2) a proximity function that measures how far a state deviates from expert behavior and provides corrective guidance during exploration.
  arXiv  Detail & Related papers  (2026-07-20T09:51:02Z)
• Stage-Transition Dense Reward Modeling for Reinforcement Learning [22.634812963829194]
  Reinforcement learning for long-horizon robotic manipulation is often limited by sparse and delayed rewards.<n>This work proposes Stage-Transition Dense Reward (STDR), a visual reward-learning framework that converts unstructured expert videos into logically grounded dense rewards for training RL agents from scratch.<n>STDR delivers two complementary learning signals during online training: (i) stage-transition feedback that provides goal-directed reward, and (ii) within-stage progress feedback that supplies fine-grained guidance toward completing each stage.
  arXiv  Detail & Related papers  (2026-06-30T09:07:19Z)
• SENTINEL: Failure-Driven Reinforcement Learning for Training Tool-Using Language Model Agents [52.758563996979596]
  Language model agents are increasingly effective in solving realistic tasks through multi-turn tool use.<n>We propose SENTINEL, a failure-driven reinforcement learning framework that turns rollout failures into targeted training tasks.<n>On Tau2-Bench Retail with Qwen3-4B-Thinking-2507, SENTINEL improves Pass1 from 66.4 to 74.9 and outperforms RL on general synthetic tasks across Passk metrics.
  arXiv  Detail & Related papers  (2026-06-11T05:06:50Z)
• ProgVLA: Progress-Aware Robot Manipulation Skill Learning [13.001629637153583]
  We present ProgVLA, a compact vision-language-action (VLA) model for reliable robot manipulation under tight compute and memory budgets.<n>A multi-modal encoder with a two-stage Perceiver resampling scheme compresses variable-length visual, language, and proprioceptive streams into a fixed set of control-ready context tokens.<n>An auxiliary set of progress heads is trained with offline reinforcement learning objectives to jointly learn critics over normalized remaining-horizon targets.
  arXiv  Detail & Related papers  (2026-05-27T09:44:46Z)
• CORE: Code-based Inverse Self-Training Framework with Graph Expansion for Virtual Agents [69.88668127604875]
  We present CORE, a Code-based Inverse Self-Training Framework with Graph Expansion.<n> CORE bridges imitation and exploration, offering a novel training framework that promotes behavioral diversity.<n> Experiments on Web and Android platforms demonstrate that CORE significantly improves both overall performance and generalization.
  arXiv  Detail & Related papers  (2026-01-05T15:24:05Z)
• Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Models [63.69856480318313]
  AGILE formulates jigsaw solving as an interactive process, enabling the model to progressively engage with the environment.<n>We show that AGILE substantially boosts performance on jigsaw tasks of varying complexity.<n>We also demonstrate strong generalization across 9 general vision tasks, achieving an average improvement of 3.1%.
  arXiv  Detail & Related papers  (2025-10-01T17:58:05Z)
• Subtask-Aware Visual Reward Learning from Segmented Demonstrations [97.80917991633248]
  This paper introduces REDS: REward learning from Demonstration with Demonstrations, a novel reward learning framework.<n>We train a dense reward function conditioned on video segments and their corresponding subtasks to ensure alignment with ground-truth reward signals.<n>Our experiments show that REDS significantly outperforms baseline methods on complex robotic manipulation tasks in Meta-World.
  arXiv  Detail & Related papers  (2025-02-28T01:25:37Z)
• MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learning [17.437573206368494]
  Visual deep reinforcement learning (RL) enables robots to acquire skills from visual input for unstructured tasks.<n>We present MENTOR, a method that improves both the architecture and optimization of RL agents.<n>MenTOR outperforms state-of-the-art methods across three simulation benchmarks and achieves an average of 83% success rate on three challenging real-world robotic manipulation tasks.
  arXiv  Detail & Related papers  (2024-10-19T04:31:54Z)
• Affordance-Guided Reinforcement Learning via Visual Prompting [51.361977466993345]
  Keypoint-based Affordance Guidance for Improvements (KAGI) is a method leveraging rewards shaped by vision-language models (VLMs) for autonomous RL.<n>On diverse real-world manipulation tasks specified by natural language descriptions, KAGI improves the sample efficiency of autonomous RL and enables successful task completion in 30K online fine-tuning steps.
  arXiv  Detail & Related papers  (2024-07-14T21:41:29Z)
• RILe: Reinforced Imitation Learning [60.63173816209543]
  RILe (Reinforced Learning) is a framework that combines the strengths of imitation learning and inverse reinforcement learning to learn a dense reward function efficiently.<n>Our framework produces high-performing policies in high-dimensional tasks where direct imitation fails to replicate complex behaviors.
  arXiv  Detail & Related papers  (2024-06-12T17:56:31Z)
• Latent Policies for Adversarial Imitation Learning [21.105328282702885]
  This paper considers learning robot locomotion and manipulation tasks from expert demonstrations. Generative adversarial imitation learning (GAIL) trains a discriminator that distinguishes expert from agent transitions, and in turn use a reward defined by the discriminator output to optimize a policy generator for the agent. A key insight of this work is that performing imitation learning in a suitable latent task space makes the training process stable, even in challenging high-dimensional problems.
  arXiv  Detail & Related papers  (2022-06-22T18:06:26Z)
• Bridging the Imitation Gap by Adaptive Insubordination [88.35564081175642]
  We show that when the teaching agent makes decisions with access to privileged information, this information is marginalized during imitation learning. We propose 'Adaptive Insubordination' (ADVISOR) to address this gap. ADVISOR dynamically weights imitation and reward-based reinforcement learning losses during training, enabling on-the-fly switching between imitation and exploration.
  arXiv  Detail & Related papers  (2020-07-23T17:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.