Phase-Conditioned Imitation Learning with Autonomous Failure Recovery for Robust Deformable Object Manipulation

• URL: http://arxiv.org/abs/2605.29407v1
• Date: Thu, 28 May 2026 05:59:53 GMT
• Title: Phase-Conditioned Imitation Learning with Autonomous Failure Recovery for Robust Deformable Object Manipulation
• Abstract summary: We present a phase-conditioned, force-aware framework for deformable object manipulation.<n>A FiLM-conditioned ACT encoder modulates feature extraction based on the current task phase.<n>A multi-modal phase predictor fusing visual, force, and pose feedback estimates the phase in real time, detecting contact failures that are invisible to vision alone.
• Score: 5.524438437366922
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper presents a phase-conditioned, force-aware framework for robust deformable object manipulation. Standard imitation learning policies such as Action Chunking with Transformers (ACT) rely on a Markovian assumption at inference, causing state aliasing when visually similar observations require contradictory actions and preventing autonomous recovery from execution failures. We address this with a closed-loop hierarchical architecture. A FiLM-conditioned ACT encoder modulates feature extraction based on the current task phase, enabling a single unified policy to produce phase-specific behaviors while sharing action dynamics across phases. A multi-modal phase predictor fusing visual, force, and pose feedback estimates the phase in real time, detecting contact failures that are invisible to vision alone and autonomously triggering recovery trajectories. The system is completed by a hybrid impedance controller for compliant execution and a haptic teleoperation interface for force-aware data collection. Ablation studies show that FiLM-based modulation significantly outperforms both unconditioned and token-level conditioned baselines, and t-SNE analysis confirms that FiLM induces well-separated, phase-specific feature representations. Validated on hanging and removing a T-shirt with dual arms, the closed-loop system improves the hanging success rate from 56\% to 87\% through autonomous error recovery. Code and videos: https://leledeyuan00.github.io/phaser/

Related papers

• Phase-and-First-Arrival VLM Feedback for Sparse-Reward Reinforcement Learning in Surgical Manipulation [18.513281440175113]
  In sparse-reward reinforcement learning, terminal rewards collapse such attempts to the same outcome.<n>We introduce phase-and-first-arrival feedback: one VLM query per recorded episode identifies the furthest visually verified task phase.<n>Across five simulated tasks, our method reaches 75.2% mean success, compared with 52.1% for a reward based on Contrastive Language-Image Pre-training.
  arXiv  Detail & Related papers  (2026-09-07T08:29:59Z)
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution [49.776611937968]
  We present PhyAgentOS, a foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services.<n>Its Session-Centered treats a session, not an action, as the minimum unit of scheduling, compatibility, supervised execution, evidence collection, and acceptance.<n>SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan.<n> Benchmarking reuses the deployment session and verification path, so results trace to real execution.
  arXiv  Detail & Related papers  (2026-07-18T04:46:53Z)
• DeCoDrift: Stabilizing Decoder Coupling in Closed-Loop Foundation Segmentation [0.431493144418712]
  We show that feedback loop can induce a previously overlooked failure mode, decoder coupling drift, in which the mask decoder's cross-attention progressively loses alignment with the target object, causing errors to accumulate across iterations.<n>We introduce DeCoDrift, a training-free inference-time stabilization framework that constrains prompt updates and preserves decoder coupling across iterations.
  arXiv  Detail & Related papers  (2026-05-25T11:41:21Z)
• VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model [54.35791816657227]
  We introduce textbfVLA-ATTC', a framework that endows VLA models with adaptive test-time compute.<n>VLA-ATTC employs an uncertainty-based cognitive clutch'' to dynamically transition from reflexive execution to a TTC deliberation phase.<n>On the LIBERO-LONG benchmark, VLA-ATTC reduces the failure rate of the SOTA model PI0.5 by over 50%.
  arXiv  Detail & Related papers  (2026-05-02T02:13:11Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• Rewind-IL: Online Failure Detection and State Respawning for Imitation Learning [7.445072780282545]
  We present Rewind-IL, a training-free online safeguard framework for generative action-chunked imitation policies.<n> Rewind-IL combines a zero-shot failure detector based on Temporal Inter-chunk Discrepancy Estimate (TIDE), with split conformal prediction, with a state-respawning mechanism.<n>Online, Rewind-IL monitors self-consistency in overlapping action chunks, tracks similarity to the checkpoint library, and, upon failure, rewinds execution to the latest verified safe state.
  arXiv  Detail & Related papers  (2026-04-17T20:41:14Z)
• A Physical Agentic Loop for Language-Guided Grasping with Execution-State Monitoring [5.437966695589128]
  We reformulate language-guided grasping as a bounded embodied agent operating over grounded execution states.<n>We introduce a physical agentic loop that wraps an unmodified learned manipulation primitive.<n>We validate the resulting loop on a mobile manipulator with an eye-in-hand D405 camera.
  arXiv  Detail & Related papers  (2026-04-08T08:01:35Z)
• Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards [76.49428173793386]
  LLMs must invoke multiple dependent APIs in the correct order while propagating intermediate outputs.<n>Training models to handle such faces two obstacles: existing environments focus on simple per-turn function calls with simulated data, and binary rewards.<n>We present a framework addressing both challenges. First, we construct a reinforcement learning environment backed by a large-scale cache of real API responses, enabling a data synthesis pipeline that samples valid multi-step orchestration traces.<n>Second, we propose a graduated reward design that decomposes correctness into atomic validity and orchestration.
  arXiv  Detail & Related papers  (2026-03-25T18:31:39Z)
• See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation [59.07792608884117]
  We introduce See, Plan, Rewind (SPR), a progress-aware vision-language-action framework.<n>SPR operates through a continuous core cycle, Seeing the current state and upcoming milestone, Planning a trajectory towards the next 2D waypoint, and Rewinding to a recoverable state upon failure.<n>SPR achieves state-of-the-art robustness with the smallest performance drop, surpassing OpenVLA-OFT and UniVLA.
  arXiv  Detail & Related papers  (2026-03-10T07:22:51Z)
• Mind the Gap: Learning Implicit Impedance in Visuomotor Policies via Intent-Execution Mismatch [7.078279704479455]
  We propose a Dual-State Conditioning framework that shifts the learning objective to "Intent Cloning" (master command)<n>By predicting the master intent, our policy learns to generate a "virtual equilibrium point", effectively realizing implicit impedance control.<n>This presents a minimalist behavior cloning framework for low-cost hardware, enabling force perception and dynamic compensation without relying on explicit force sensing.
  arXiv  Detail & Related papers  (2026-02-09T15:18:12Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Model [63.336123527432136]
  We introduce Bench2Drive-R, a generative framework that enables reactive closed-loop evaluation.<n>Unlike existing video generative models for autonomous driving, the proposed designs are tailored for interactive simulation.<n>We compare the generation quality of Bench2Drive-R with existing generative models and achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-11T06:35:18Z)
• PIDformer: Transformer Meets Control Theory [28.10913642120948]
  We unveil self-attention as an autonomous state-space model that inherently promotes smoothness in its solutions. We incorporate a Proportional-Integral-Derivative (PID) closed-loop feedback control system with a reference point into the model to improve robustness and representation capacity. Motivated by this control framework, we derive a novel class of transformers, PID-controlled Transformer (PIDformer)
  arXiv  Detail & Related papers  (2024-02-25T05:04:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.