Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models

• URL: http://arxiv.org/abs/2105.12724v1
• Date: Wed, 26 May 2021 17:57:19 GMT
• Title: Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models
• Authors: Boyuan Chen, Yuhang Hu, Lianfeng Li, Sara Cummings, Hod Lipson
• Abstract summary: Ability to generate intelligent and generalizable facial expressions is essential for building human-like social robots. We develop a vision-based self-supervised learning framework for facial mimicry. Our method enables accurate and diverse face mimicry across diverse human subjects.
• Score: 11.925808365657936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ability to generate intelligent and generalizable facial expressions is essential for building human-like social robots. At present, progress in this field is hindered by the fact that each facial expression needs to be programmed by humans. In order to adapt robot behavior in real time to different situations that arise when interacting with human subjects, robots need to be able to train themselves without requiring human labels, as well as make fast action decisions and generalize the acquired knowledge to diverse and new contexts. We addressed this challenge by designing a physical animatronic robotic face with soft skin and by developing a vision-based self-supervised learning framework for facial mimicry. Our algorithm does not require any knowledge of the robot's kinematic model, camera calibration or predefined expression set. By decomposing the learning process into a generative model and an inverse model, our framework can be trained using a single motor babbling dataset. Comprehensive evaluations show that our method enables accurate and diverse face mimicry across diverse human subjects. The project website is at http://www.cs.columbia.edu/~bchen/aiface/

Related papers

• $π_0$: A Vision-Language-Action Flow Model for General Robot Control [77.32743739202543]
  We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people, and its ability to acquire new skills via fine-tuning.
  arXiv  Detail & Related papers  (2024-10-31T17:22:30Z)
• HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation [50.616995671367704]
  We present a high-dimensional, simulated robot learning benchmark, HumanoidBench, featuring a humanoid robot equipped with dexterous hands. Our findings reveal that state-of-the-art reinforcement learning algorithms struggle with most tasks, whereas a hierarchical learning approach achieves superior performance when supported by robust low-level policies.
  arXiv  Detail & Related papers  (2024-03-15T17:45:44Z)
• Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Plans [58.27029676638521]
  We show how passive human videos can serve as a rich source of data for learning such generalist robots. We learn a human plan predictor that, given a current image of a scene and a goal image, predicts the future hand and object configurations. We show that our learned system can perform over 16 manipulation skills that generalize to 40 objects.
  arXiv  Detail & Related papers  (2023-12-01T18:54:12Z)
• Robot Learning with Sensorimotor Pre-training [98.7755895548928]
  We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
  arXiv  Detail & Related papers  (2023-06-16T17:58:10Z)
• Affordances from Human Videos as a Versatile Representation for Robotics [31.248842798600606]
  We train a visual affordance model that estimates where and how in the scene a human is likely to interact. The structure of these behavioral affordances directly enables the robot to perform many complex tasks. We show the efficacy of our approach, which we call VRB, across 4 real world environments, over 10 different tasks, and 2 robotic platforms operating in the wild.
  arXiv  Detail & Related papers  (2023-04-17T17:59:34Z)
• Open-World Object Manipulation using Pre-trained Vision-Language Models [72.87306011500084]
  For robots to follow instructions from people, they must be able to connect the rich semantic information in human vocabulary. We develop a simple approach, which leverages a pre-trained vision-language model to extract object-identifying information. In a variety of experiments on a real mobile manipulator, we find that MOO generalizes zero-shot to a wide range of novel object categories and environments.
  arXiv  Detail & Related papers  (2023-03-02T01:55:10Z)
• Towards Modeling and Influencing the Dynamics of Human Learning [26.961274302321343]
  We take a step towards enabling robots to understand the influence they have, leverage it to better assist people, and help human models more quickly align with reality. Our key idea is to model the human's learning as a nonlinear dynamical system which evolves the human's internal model given new observations. We then formalize how robots can influence human learning by embedding the human's learning dynamics model into the robot planning problem.
  arXiv  Detail & Related papers  (2023-01-02T23:59:45Z)
• HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration [57.045140028275036]
  We show that manipulation skills can be transferred from a human to a robot through the use of micro-evolutionary reinforcement learning. We propose an algorithm for multi-dimensional evolution path searching that allows joint optimization of both the robot evolution path and the policy.
  arXiv  Detail & Related papers  (2022-12-08T15:56:13Z)
• Full-Body Visual Self-Modeling of Robot Morphologies [29.76701883250049]
  Internal computational models of physical bodies are fundamental to the ability of robots and animals alike to plan and control their actions. Recent progress in fully data-driven self-modeling has enabled machines to learn their own forward kinematics directly from task-agnostic interaction data. Here, we propose that instead of directly modeling forward-kinematics, a more useful form of self-modeling is one that could answer space occupancy queries.
  arXiv  Detail & Related papers  (2021-11-11T18:58:07Z)
• Neuroscience-inspired perception-action in robotics: applying active inference for state estimation, control and self-perception [2.1067139116005595]
  We discuss how neuroscience findings open up opportunities to improve current estimation and control algorithms in robotics. This paper summarizes some experiments and lessons learned from developing such a computational model on real embodied platforms.
  arXiv  Detail & Related papers  (2021-05-10T10:59:38Z)
• A Survey of Behavior Learning Applications in Robotics -- State of the Art and Perspectives [44.45953630612019]
  Recent success of machine learning in many domains has been overwhelming. We will give a broad overview of behaviors that have been learned and used on real robots.
  arXiv  Detail & Related papers  (2019-06-05T07:54:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.