Learning to Shift Attention for Motion Generation
- URL: http://arxiv.org/abs/2102.12141v1
- Date: Wed, 24 Feb 2021 09:07:52 GMT
- Title: Learning to Shift Attention for Motion Generation
- Authors: You Zhou and Jianfeng Gao and Tamim Asfour
- Abstract summary: One challenge of motion generation using robot learning from demonstration techniques is that human demonstrations follow a distribution with multiple modes for one task query.
Previous approaches fail to capture all modes or tend to average modes of the demonstrations and thus generate invalid trajectories.
We propose a motion generation model with extrapolation ability to overcome this problem.
- Score: 55.61994201686024
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: One challenge of motion generation using robot learning from demonstration
techniques is that human demonstrations follow a distribution with multiple
modes for one task query. Previous approaches fail to capture all modes or tend
to average modes of the demonstrations and thus generate invalid trajectories.
The other difficulty is the small number of demonstrations that cannot cover
the entire working space. To overcome this problem, a motion generation model
with extrapolation ability is needed. Previous works restrict task queries as
local frames and learn representations in local frames. We propose a model to
solve both problems. For multiple modes, we suggest to learn local latent
representations of motion trajectories with a density estimation method based
on real-valued non-volume preserving (RealNVP) transformations that provides a
set of powerful, stably invertible, and learnable transformations. To improve
the extrapolation ability, we propose to shift the attention of the robot from
one local frame to another during the task execution. In experiments, we
consider the docking problem used also in previous works where a trajectory has
to be generated to connect two dockers without collision. We increase
complexity of the task and show that the proposed method outperforms other
approaches. In addition, we evaluate the approach in real robot experiments.
Related papers
- Affordance-based Robot Manipulation with Flow Matching [6.863932324631107]
Our framework unifies affordance model learning and trajectory generation with flow matching for robot manipulation.
Our evaluation highlights that the proposed prompt tuning method for learning manipulation affordance with language prompter achieves competitive performance.
Our framework seamlessly unifies affordance model learning and trajectory generation with flow matching for robot manipulation.
arXiv Detail & Related papers (2024-09-02T09:11:28Z) - Deciphering Movement: Unified Trajectory Generation Model for Multi-Agent [53.637837706712794]
We propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs.
Specifically, we introduce a Ghost Spatial Masking (GSM) module embedded within a Transformer encoder for spatial feature extraction.
We benchmark three practical sports game datasets, Basketball-U, Football-U, and Soccer-U, for evaluation.
arXiv Detail & Related papers (2024-05-27T22:15:23Z) - DITTO: Demonstration Imitation by Trajectory Transformation [31.930923345163087]
In this work, we address the problem of one-shot imitation from a single human demonstration, given by an RGB-D video recording.
We propose a two-stage process. In the first stage we extract the demonstration trajectory offline. This entails segmenting manipulated objects and determining their relative motion in relation to secondary objects such as containers.
In the online trajectory generation stage, we first re-detect all objects, then warp the demonstration trajectory to the current scene and execute it on the robot.
arXiv Detail & Related papers (2024-03-22T13:46:51Z) - One ACT Play: Single Demonstration Behavior Cloning with Action Chunking
Transformers [11.875194596371484]
Humans can learn to complete tasks, even complex ones, after only seeing one or two demonstrations.
Our work seeks to emulate this ability, using behavior cloning to learn a task given only a single human demonstration.
We develop a novel addition to the temporal ensembling method used by action chunking agents during inference.
arXiv Detail & Related papers (2023-09-18T21:50:26Z) - Learning Reward Functions for Robotic Manipulation by Observing Humans [92.30657414416527]
We use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies.
The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective.
arXiv Detail & Related papers (2022-11-16T16:26:48Z) - Eliciting Compatible Demonstrations for Multi-Human Imitation Learning [16.11830547863391]
Imitation learning from human-provided demonstrations is a strong approach for learning policies for robot manipulation.
Natural human behavior has a great deal of heterogeneity, with several optimal ways to demonstrate a task.
This mismatch presents a problem for interactive imitation learning, where sequences of users improve on a policy by iteratively collecting new, possibly conflicting demonstrations.
We show that we can both identify incompatible demonstrations via post-hoc filtering, and apply our compatibility measure to actively elicit compatible demonstrations from new users.
arXiv Detail & Related papers (2022-10-14T19:37:55Z) - Human-in-the-Loop Imitation Learning using Remote Teleoperation [72.2847988686463]
We build a data collection system tailored to 6-DoF manipulation settings.
We develop an algorithm to train the policy iteratively on new data collected by the system.
We demonstrate that agents trained on data collected by our intervention-based system and algorithm outperform agents trained on an equivalent number of samples collected by non-interventional demonstrators.
arXiv Detail & Related papers (2020-12-12T05:30:35Z) - Visual Imitation Made Easy [102.36509665008732]
We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots.
We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector.
We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
arXiv Detail & Related papers (2020-08-11T17:58:50Z) - Human Motion Transfer from Poses in the Wild [61.6016458288803]
We tackle the problem of human motion transfer, where we synthesize novel motion video for a target person that imitates the movement from a reference video.
It is a video-to-video translation task in which the estimated poses are used to bridge two domains.
We introduce a novel pose-to-video translation framework for generating high-quality videos that are temporally coherent even for in-the-wild pose sequences unseen during training.
arXiv Detail & Related papers (2020-04-07T05:59:53Z) - Meta Adaptation using Importance Weighted Demonstrations [19.37671674146514]
In some cases, the distribution shifts, so much, that it is difficult for an agent to infer the new task.
We propose a novel algorithm to generalize on any related task by leveraging prior knowledge on a set of specific tasks.
We show experiments where the robot is trained from a diversity of environmental tasks and is also able to adapt to an unseen environment.
arXiv Detail & Related papers (2019-11-23T07:22:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.