Reinforcement Learning of Musculoskeletal Control from Functional Simulations

• URL: http://arxiv.org/abs/2007.06669v1
• Date: Mon, 13 Jul 2020 20:20:01 GMT
• Title: Reinforcement Learning of Musculoskeletal Control from Functional Simulations
• Authors: Emanuel Joos, Fabien P\'ean, Orcun Goksel
• Abstract summary: In this work, a deep reinforcement learning (DRL) based inverse dynamics controller is trained to control muscle activations of a biomechanical model of the human shoulder. Results are presented for a single-axis motion control of shoulder abduction for the task of following randomly generated angular trajectories.
• Score: 3.94716580540538
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To diagnose, plan, and treat musculoskeletal pathologies, understanding and reproducing muscle recruitment for complex movements is essential. With muscle activations for movements often being highly redundant, nonlinear, and time dependent, machine learning can provide a solution for their modeling and control for anatomy-specific musculoskeletal simulations. Sophisticated biomechanical simulations often require specialized computational environments, being numerically complex and slow, hindering their integration with typical deep learning frameworks. In this work, a deep reinforcement learning (DRL) based inverse dynamics controller is trained to control muscle activations of a biomechanical model of the human shoulder. In a generalizable end-to-end fashion, muscle activations are learned given current and desired position-velocity pairs. A customized reward functions for trajectory control is introduced, enabling straightforward extension to additional muscles and higher degrees of freedom. Using the biomechanical model, multiple episodes are simulated on a cluster simultaneously using the evolving neural models of the DRL being trained. Results are presented for a single-axis motion control of shoulder abduction for the task of following randomly generated angular trajectories.

Related papers

• Muscles in Time: Learning to Understand Human Motion by Simulating Muscle Activations [64.98299559470503]
  Muscles in Time (MinT) is a large-scale synthetic muscle activation dataset. It contains over nine hours of simulation data covering 227 subjects and 402 simulated muscle strands. We show results on neural network-based muscle activation estimation from human pose sequences.
  arXiv  Detail & Related papers  (2024-10-31T18:28:53Z)
• MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints [50.61346764110482]
  We integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We also propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron network.
  arXiv  Detail & Related papers  (2024-04-16T02:18:18Z)
• A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamics [73.35846234413611]
  In drug discovery, molecular dynamics (MD) simulation provides a powerful tool for predicting binding affinities, estimating transport properties, and exploring pocket sites. We propose NeuralMD, the first machine learning (ML) surrogate that can facilitate numerical MD and provide accurate simulations in protein-ligand binding dynamics. We demonstrate the efficiency and effectiveness of NeuralMD, achieving over 1K$times$ speedup compared to standard numerical MD simulations.
  arXiv  Detail & Related papers  (2024-01-26T09:35:17Z)
• Natural and Robust Walking using Reinforcement Learning without Demonstrations in High-Dimensional Musculoskeletal Models [29.592874007260342]
  Humans excel at robust bipedal walking in complex natural environments. It is still not fully understood how the nervous system resolves the musculoskeletal redundancy to solve the multi-objective control problem.
  arXiv  Detail & Related papers  (2023-09-06T13:20:31Z)
• MyoSuite -- A contact-rich simulation suite for musculoskeletal motor control [7.856809409051587]
  MyoSuite is a suite of physiologically accurate biomechanical models of elbow, wrist, and hand, with physical contact capabilities. We provide diverse motor-control challenges: from simple postural control to skilled hand-object interactions.
  arXiv  Detail & Related papers  (2022-05-26T20:11:23Z)
• From Motion to Muscle [0.0]
  We show that muscle activity can be artificially generated based on motion features such as position, velocity, and acceleration. The model achieves remarkable precision for previously trained movements and maintains significantly high precision for new movements that have not been previously trained.
  arXiv  Detail & Related papers  (2022-01-27T13:30:17Z)
• OstrichRL: A Musculoskeletal Ostrich Simulation to Study Bio-mechanical Locomotion [8.849771760994273]
  We release a 3D musculoskeletal simulation of an ostrich based on the MuJoCo simulator. The model is based on CT scans and dissections used to gather actual muscle data. We also provide a set of reinforcement learning tasks, including reference motion tracking and a reaching task with the neck.
  arXiv  Detail & Related papers  (2021-12-11T19:58:11Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)
• Learning Control Policies for Imitating Human Gaits [2.28438857884398]
  Humans exhibit movements like walking, running, and jumping in the most efficient manner, which served as the source of motivation for this project. Skeletal and Musculoskeletal human models were considered for motions in the sagittal plane. Model-free reinforcement learning algorithms were used to optimize inverse dynamics control actions.
  arXiv  Detail & Related papers  (2021-05-15T16:33:24Z)
• Continuous Learning and Adaptation with Membrane Potential and Activation Threshold Homeostasis [91.3755431537592]
  This paper presents the Membrane Potential and Activation Threshold Homeostasis (MPATH) neuron model. The model allows neurons to maintain a form of dynamic equilibrium by automatically regulating their activity when presented with input. Experiments demonstrate the model's ability to adapt to and continually learn from its input.
  arXiv  Detail & Related papers  (2021-04-22T04:01:32Z)
• GEM: Group Enhanced Model for Learning Dynamical Control Systems [78.56159072162103]
  We build effective dynamical models that are amenable to sample-based learning. We show that learning the dynamics on a Lie algebra vector space is more effective than learning a direct state transition model. This work sheds light on a connection between learning of dynamics and Lie group properties, which opens doors for new research directions.
  arXiv  Detail & Related papers  (2021-04-07T01:08:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.