Realtime Motion Generation with Active Perception Using Attention
Mechanism for Cooking Robot
- URL: http://arxiv.org/abs/2309.14837v1
- Date: Tue, 26 Sep 2023 11:05:37 GMT
- Title: Realtime Motion Generation with Active Perception Using Attention
Mechanism for Cooking Robot
- Authors: Namiko Saito, Mayu Hiramoto, Ayuna Kubo, Kanata Suzuki, Hiroshi Ito,
Shigeki Sugano and Tetsuya Ogata
- Abstract summary: We tackle the task of cooking scrambled eggs using real ingredients.
The robot needs to perceive the states of the egg and adjust stirring movement in real time.
We propose a predictive recurrent neural network with an attention mechanism that can weigh the sensor input.
- Score: 9.186595261712974
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: To support humans in their daily lives, robots are required to autonomously
learn, adapt to objects and environments, and perform the appropriate actions.
We tackled on the task of cooking scrambled eggs using real ingredients, in
which the robot needs to perceive the states of the egg and adjust stirring
movement in real time, while the egg is heated and the state changes
continuously. In previous works, handling changing objects was found to be
challenging because sensory information includes dynamical, both important or
noisy information, and the modality which should be focused on changes every
time, making it difficult to realize both perception and motion generation in
real time. We propose a predictive recurrent neural network with an attention
mechanism that can weigh the sensor input, distinguishing how important and
reliable each modality is, that realize quick and efficient perception and
motion generation. The model is trained with learning from the demonstration,
and allows the robot to acquire human-like skills. We validated the proposed
technique using the robot, Dry-AIREC, and with our learning model, it could
perform cooking eggs with unknown ingredients. The robot could change the
method of stirring and direction depending on the status of the egg, as in the
beginning it stirs in the whole pot, then subsequently, after the egg started
being heated, it starts flipping and splitting motion targeting specific areas,
although we did not explicitly indicate them.
Related papers
- Naturalistic Robot Arm Trajectory Generation via Representation Learning [4.7682079066346565]
Integration of manipulator robots in household environments suggests a need for more predictable human-like robot motion.
One method of generating naturalistic motion trajectories is via imitation of human demonstrators.
This paper explores a self-supervised imitation learning method using an autoregressive neural network for an assistive drinking task.
arXiv Detail & Related papers (2023-09-14T09:26:03Z) - Robot Learning with Sensorimotor Pre-training [98.7755895548928]
We present a self-supervised sensorimotor pre-training approach for robotics.
Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens.
We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
arXiv Detail & Related papers (2023-06-16T17:58:10Z) - Cross-Domain Transfer via Semantic Skill Imitation [49.83150463391275]
We propose an approach for semantic imitation, which uses demonstrations from a source domain, e.g. human videos, to accelerate reinforcement learning (RL)
Instead of imitating low-level actions like joint velocities, our approach imitates the sequence of demonstrated semantic skills like "opening the microwave" or "turning on the stove"
arXiv Detail & Related papers (2022-12-14T18:46:14Z) - HERD: Continuous Human-to-Robot Evolution for Learning from Human
Demonstration [57.045140028275036]
We show that manipulation skills can be transferred from a human to a robot through the use of micro-evolutionary reinforcement learning.
We propose an algorithm for multi-dimensional evolution path searching that allows joint optimization of both the robot evolution path and the policy.
arXiv Detail & Related papers (2022-12-08T15:56:13Z) - Robot Cooking with Stir-fry: Bimanual Non-prehensile Manipulation of
Semi-fluid Objects [13.847796949856457]
This letter describes an approach to achieve well-known Chinese cooking art stir-fry on a bimanual robot system.
We define a canonical stir-fry movement, then propose a decoupled framework for learning deformable object manipulation from human demonstration.
By adding visual feedback, our framework can adjust the movements automatically to achieve the desired stir-fry effect.
arXiv Detail & Related papers (2022-05-12T08:58:30Z) - Synthesis and Execution of Communicative Robotic Movements with
Generative Adversarial Networks [59.098560311521034]
We focus on how to transfer on two different robotic platforms the same kinematics modulation that humans adopt when manipulating delicate objects.
We choose to modulate the velocity profile adopted by the robots' end-effector, inspired by what humans do when transporting objects with different characteristics.
We exploit a novel Generative Adversarial Network architecture, trained with human kinematics examples, to generalize over them and generate new and meaningful velocity profiles.
arXiv Detail & Related papers (2022-03-29T15:03:05Z) - From Movement Kinematics to Object Properties: Online Recognition of
Human Carefulness [112.28757246103099]
We show how a robot can infer online, from vision alone, whether or not the human partner is careful when moving an object.
We demonstrated that a humanoid robot could perform this inference with high accuracy (up to 81.3%) even with a low-resolution camera.
The prompt recognition of movement carefulness from observing the partner's action will allow robots to adapt their actions on the object to show the same degree of care as their human partners.
arXiv Detail & Related papers (2021-09-01T16:03:13Z) - Learning Bipedal Robot Locomotion from Human Movement [0.791553652441325]
We present a reinforcement learning based method for teaching a real world bipedal robot to perform movements directly from motion capture data.
Our method seamlessly transitions from training in a simulation environment to executing on a physical robot.
We demonstrate our method on an internally developed humanoid robot with movements ranging from a dynamic walk cycle to complex balancing and waving.
arXiv Detail & Related papers (2021-05-26T00:49:37Z) - Learning Agile Robotic Locomotion Skills by Imitating Animals [72.36395376558984]
Reproducing the diverse and agile locomotion skills of animals has been a longstanding challenge in robotics.
We present an imitation learning system that enables legged robots to learn agile locomotion skills by imitating real-world animals.
arXiv Detail & Related papers (2020-04-02T02:56:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.