Feasibility-aware Imitation Learning from Observation with Multimodal Feedback

• URL: http://arxiv.org/abs/2602.15351v1
• Date: Tue, 17 Feb 2026 04:34:29 GMT
• Title: Feasibility-aware Imitation Learning from Observation with Multimodal Feedback
• Abstract summary: We propose Feasibility-Aware Behavior Cloning from Observation (FABCO)<n>FABCO integrates behavior cloning from observation, which complements robot actions using robot dynamics models, with feasibility estimation.<n>We conducted experiments with 15 participants on two tasks and confirmed that FABCO improves imitation learning performance by more than 3.2 times.
• Score: 4.374284509621931
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imitation learning frameworks that learn robot control policies from demonstrators' motions via hand-mounted demonstration interfaces have attracted increasing attention. However, due to differences in physical characteristics between demonstrators and robots, this approach faces two limitations: i) the demonstration data do not include robot actions, and ii) the demonstrated motions may be infeasible for robots. These limitations make policy learning difficult. To address them, we propose Feasibility-Aware Behavior Cloning from Observation (FABCO). FABCO integrates behavior cloning from observation, which complements robot actions using robot dynamics models, with feasibility estimation. In feasibility estimation, the demonstrated motions are evaluated using a robot-dynamics model, learned from the robot's execution data, to assess reproducibility under the robot's dynamics. The estimated feasibility is used for multimodal feedback and feasibility-aware policy learning to improve the demonstrator's motions and learn robust policies. Multimodal feedback provides feasibility through the demonstrator's visual and haptic senses to promote feasible demonstrated motions. Feasibility-aware policy learning reduces the influence of demonstrated motions that are infeasible for robots, enabling the learning of policies that robots can execute stably. We conducted experiments with 15 participants on two tasks and confirmed that FABCO improves imitation learning performance by more than 3.2 times compared to the case without feasibility feedback.

Related papers

• Learning Physical Interaction: A Survey of Tactile- and Force-aware Robot Learning [128.76135832773102]
  Physically grounded robot intelligence requires robots to perceive, reason about, and regulate their interactions with the physical world.<n>Recent robot learning methods have made substantial progress by integrating force, tactile, vision, language, and proprioceptive sensing into learned manipulation policies.<n>Despite these advances, existing surveys have not explicitly reviewed force- and tactile-aware robot learning from a unified perspective.
  arXiv  Detail & Related papers  (2026-08-02T14:22:01Z)
• Empathetic Motion Generation for Humanoid Educational Robots via Reasoning-Guided Vision--Language--Motion Diffusion Architecture [4.77320559246954]
  This article suggests a reasoning-guided vision-language-motion diffusion framework (RG-VLMD) for generating instruction-aware co-speech gestures for humanoid robots in educational scenarios.<n>The system integrates multi-modal affective estimation, pedagogical reasoning, and teaching-act-conditioned motion synthesis to enable adaptive and semantically consistent robot behavior.
  arXiv  Detail & Related papers  (2026-03-19T11:26:04Z)
• Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations [76.79742393097358]
  Vision-Language Action (VLAs) models promise to extend the remarkable success of vision-language models (VLMs) to robotics.<n>Existing fine-tuning methods lack specificity, adapting the same set of parameters regardless of a task's visual, linguistic, and physical characteristics.<n>Inspired by functional specificity in neuroscience, we hypothesize that it is more effective to finetune sparse model representations specific to a given task.
  arXiv  Detail & Related papers  (2025-11-27T18:50:21Z)
• Feasibility-aware Imitation Learning from Observations through a Hand-mounted Demonstration Interface [10.808201018448274]
  We propose feasibility-aware behavior cloning from observation (FABCO)<n>In the FABCO framework, the feasibility of each demonstration is assessed using the robot's pre-trained forward and inverse dynamics models.<n>We experimentally validated FABCO's effectiveness by applying it to a pipette insertion task involving a pipette and a vial.
  arXiv  Detail & Related papers  (2025-03-12T03:14:04Z)
• Reciprocal Learning of Intent Inferral with Augmented Visual Feedback for Stroke [2.303526979876375]
  We propose a bidirectional paradigm that facilitates human adaptation to an intent inferral classifier.<n>We demonstrate this paradigm in the context of controlling a robotic hand orthosis for stroke.<n>Our experiments with stroke subjects show reciprocal learning improving performance in a subset of subjects without negatively impacting performance on the others.
  arXiv  Detail & Related papers  (2024-12-10T22:49:36Z)
• Moto: Latent Motion Token as the Bridging Language for Learning Robot Manipulation from Videos [101.26467307473638]
  We introduce Moto, which converts video content into latent Motion Token sequences by a Latent Motion Tokenizer.<n>We pre-train Moto-GPT through motion token autoregression, enabling it to capture diverse visual motion knowledge.<n>To transfer learned motion priors to real robot actions, we implement a co-fine-tuning strategy that seamlessly bridges latent motion token prediction and real robot control.
  arXiv  Detail & Related papers  (2024-12-05T18:57:04Z)
• Dynamic Non-Prehensile Object Transport via Model-Predictive Reinforcement Learning [24.079032278280447]
  We propose an approach that combines batch reinforcement learning (RL) with model-predictive control (MPC)<n>We validate the proposed approach through extensive simulated and real-world experiments on a Franka Panda robot performing the robot waiter task.
  arXiv  Detail & Related papers  (2024-11-27T03:33:42Z)
• DiffGen: Robot Demonstration Generation via Differentiable Physics Simulation, Differentiable Rendering, and Vision-Language Model [72.66465487508556]
  DiffGen is a novel framework that integrates differentiable physics simulation, differentiable rendering, and a vision-language model. It can generate realistic robot demonstrations by minimizing the distance between the embedding of the language instruction and the embedding of the simulated observation. Experiments demonstrate that with DiffGen, we could efficiently and effectively generate robot data with minimal human effort or training time.
  arXiv  Detail & Related papers  (2024-05-12T15:38:17Z)
• Unsupervised Learning of Effective Actions in Robotics [0.9374652839580183]
  Current state-of-the-art action representations in robotics lack proper effect-driven learning of the robot's actions. We propose an unsupervised algorithm to discretize a continuous motion space and generate "action prototypes" We evaluate our method on a simulated stair-climbing reinforcement learning task.
  arXiv  Detail & Related papers  (2024-04-03T13:28:52Z)
• Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning [121.9708998627352]
  Recent work has shown that, in practical robot learning applications, the effects of adversarial training do not pose a fair trade-off. This work revisits the robustness-accuracy trade-off in robot learning by analyzing if recent advances in robust training methods and theory can make adversarial training suitable for real-world robot applications.
  arXiv  Detail & Related papers  (2022-04-15T08:12:15Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.