Neural Style Transfer with Twin-Delayed DDPG for Shared Control of Robotic Manipulators

• URL: http://arxiv.org/abs/2402.00722v1
• Date: Thu, 1 Feb 2024 16:14:32 GMT
• Title: Neural Style Transfer with Twin-Delayed DDPG for Shared Control of Robotic Manipulators
• Authors: Raul Fernandez-Fernandez, Marco Aggravi, Paolo Robuffo Giordano, Juan G. Victores and Claudio Pacchierotti
• Abstract summary: We propose a framework for transferring a set of styles to the motion of a robotic manipulator. An autoencoder architecture extracts and defines the Content and the Style of the target robot motions. The proposed Neural Policy Style Transfer TD3 (NPST3) alters the robot motion by introducing the trained style.
• Score: 15.947412070402878
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Neural Style Transfer (NST) refers to a class of algorithms able to manipulate an element, most often images, to adopt the appearance or style of another one. Each element is defined as a combination of Content and Style: the Content can be conceptually defined as the what and the Style as the how of said element. In this context, we propose a custom NST framework for transferring a set of styles to the motion of a robotic manipulator, e.g., the same robotic task can be carried out in an angry, happy, calm, or sad way. An autoencoder architecture extracts and defines the Content and the Style of the target robot motions. A Twin Delayed Deep Deterministic Policy Gradient (TD3) network generates the robot control policy using the loss defined by the autoencoder. The proposed Neural Policy Style Transfer TD3 (NPST3) alters the robot motion by introducing the trained style. Such an approach can be implemented either offline, for carrying out autonomous robot motions in dynamic environments, or online, for adapting at runtime the style of a teleoperated robot. The considered styles can be learned online from human demonstrations. We carried out an evaluation with human subjects enrolling 73 volunteers, asking them to recognize the style behind some representative robotic motions. Results show a good recognition rate, proving that it is possible to convey different styles to a robot using this approach.

Related papers

• Know your limits! Optimize the robot's behavior through self-awareness [11.021217430606042]
  Recent human-robot imitation algorithms focus on following a reference human motion with high precision. We introduce a deep-learning model that anticipates the robot's performance when imitating a given reference. Our Self-AWare model (SAW) ranks potential robot behaviors based on various criteria, such as fall likelihood, adherence to the reference motion, and smoothness.
  arXiv  Detail & Related papers  (2024-09-16T14:14:58Z)
• Hand-Object Interaction Pretraining from Videos [77.92637809322231]
  We learn general robot manipulation priors from 3D hand-object interaction trajectories. We do so by sharing both the human hand and the manipulated object in 3D space and human motions to robot actions. We empirically demonstrate that finetuning this policy, with both reinforcement learning (RL) and behavior cloning (BC), enables sample-efficient adaptation to downstream tasks and simultaneously improves robustness and generalizability compared to prior approaches.
  arXiv  Detail & Related papers  (2024-09-12T17:59:07Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)
• Zero-Shot Robot Manipulation from Passive Human Videos [59.193076151832145]
  We develop a framework for extracting agent-agnostic action representations from human videos. Our framework is based on predicting plausible human hand trajectories. We deploy the trained model zero-shot for physical robot manipulation tasks.
  arXiv  Detail & Related papers  (2023-02-03T21:39:52Z)
• Robot Cooking with Stir-fry: Bimanual Non-prehensile Manipulation of Semi-fluid Objects [13.847796949856457]
  This letter describes an approach to achieve well-known Chinese cooking art stir-fry on a bimanual robot system. We define a canonical stir-fry movement, then propose a decoupled framework for learning deformable object manipulation from human demonstration. By adding visual feedback, our framework can adjust the movements automatically to achieve the desired stir-fry effect.
  arXiv  Detail & Related papers  (2022-05-12T08:58:30Z)
• Synthesis and Execution of Communicative Robotic Movements with Generative Adversarial Networks [59.098560311521034]
  We focus on how to transfer on two different robotic platforms the same kinematics modulation that humans adopt when manipulating delicate objects. We choose to modulate the velocity profile adopted by the robots' end-effector, inspired by what humans do when transporting objects with different characteristics. We exploit a novel Generative Adversarial Network architecture, trained with human kinematics examples, to generalize over them and generate new and meaningful velocity profiles.
  arXiv  Detail & Related papers  (2022-03-29T15:03:05Z)
• Counterfactual Explanation and Causal Inference in Service of Robustness in Robot Control [15.104159722499366]
  We propose an architecture for training generative models of counterfactual conditionals of the form, 'can we modify event A to cause B instead of C?' In contrast to conventional control design approaches, where robustness is quantified in terms of the ability to reject noise, we explore the space of counterfactuals that might cause a certain requirement to be violated.
  arXiv  Detail & Related papers  (2020-09-18T14:22:47Z)
• Morphology-Agnostic Visual Robotic Control [76.44045983428701]
  MAVRIC is an approach that works with minimal prior knowledge of the robot's morphology. We demonstrate our method on visually-guided 3D point reaching, trajectory following, and robot-to-robot imitation.
  arXiv  Detail & Related papers  (2019-12-31T15:45:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.