Deep learning framework for action prediction reveals multi-timescale locomotor control

• URL: http://arxiv.org/abs/2503.16340v4
• Date: Mon, 14 Apr 2025 04:08:16 GMT
• Title: Deep learning framework for action prediction reveals multi-timescale locomotor control
• Authors: Wei-Chen Wang, Antoine De Comite, Alexandra Voloshina, Monica Daley, Nidhi Seethapathi,
• Abstract summary: We develop a deep learning-based framework for action prediction.<n>We find that neural network architectures with flexible input history- timescales, like GRU and Transformer, perform best overall.
• Score: 41.985053522482545
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Modeling human movement in real-world tasks is a fundamental goal for motor control, biomechanics, and rehabilitation engineering. However, existing models of essential tasks like locomotion are not applicable across varying terrain, mechanical conditions, and sensory contexts. This is at least in part due to simplifying assumptions like linear and fixed timescales mappings between inputs and future actions, which may not be broadly applicable. Here, we develop a deep learning-based framework for action prediction, outperforming traditional models across multiple contexts (walking and running, treadmill and overground, varying terrains) and input modalities (multiple body states, visual gaze). We find that neural network architectures with flexible input history-dependence, like GRU and Transformer, and with architecture-dependent trial embeddings perform best overall. By quantifying the model's predictions relative to an autoregressive baseline, we identify context- and modality-dependent timescales. These analyses reveal that there is greater reliance on fast-timescale predictions in complex terrain, gaze predicts future foot placement before body states, and the full-body state predictions precede those by center-of-mass states. This deep learning framework for human action prediction provides quantifiable insights into the control of real-world locomotion and can be extended to other actions, contexts, and populations.

Related papers

• Unified Human Localization and Trajectory Prediction with Monocular Vision [64.19384064365431]
  MonoTransmotion is a Transformer-based framework that uses only a monocular camera to jointly solve localization and prediction tasks.<n>We show that by jointly training both tasks with our unified framework, our method is more robust in real-world scenarios made of noisy inputs.
  arXiv  Detail & Related papers  (2025-03-05T14:18:39Z)
• Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
  Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models. Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML) We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
  arXiv  Detail & Related papers  (2024-07-18T15:44:23Z)
• Adaptive Prediction Ensemble: Improving Out-of-Distribution Generalization of Motion Forecasting [15.916325272109454]
  We propose a novel framework, Adaptive Prediction Ensemble (APE), which integrates deep learning and rule-based prediction experts.<n>A learned routing function, trained concurrently with the deep learning model, dynamically selects the most reliable prediction based on the input scenario.<n>We show that our method outperforms individual prediction models and other variants, particularly in long-horizon prediction and scenarios with a high proportion of OOD data.
  arXiv  Detail & Related papers  (2024-07-12T17:57:00Z)
• Towards Generalizable and Interpretable Motion Prediction: A Deep Variational Bayes Approach [54.429396802848224]
  This paper proposes an interpretable generative model for motion prediction with robust generalizability to out-of-distribution cases. For interpretability, the model achieves the target-driven motion prediction by estimating the spatial distribution of long-term destinations. Experiments on motion prediction datasets validate that the fitted model can be interpretable and generalizable.
  arXiv  Detail & Related papers  (2024-03-10T04:16:04Z)
• Humanoid Locomotion as Next Token Prediction [84.21335675130021]
  Our model is a causal transformer trained via autoregressive prediction of sensorimotor trajectories. We show that our model enables a full-sized humanoid to walk in San Francisco zero-shot. Our model can transfer to the real world even when trained on only 27 hours of walking data, and can generalize commands not seen during training like walking backward.
  arXiv  Detail & Related papers  (2024-02-29T18:57:37Z)
• Context-based Interpretable Spatio-Temporal Graph Convolutional Network for Human Motion Forecasting [0.0]
  We present a Context- Interpretable Stemporal Graphal Network (IST-GCN) as an efficient 3D human pose forecasting model. Our architecture extracts meaningful information from pose sequences, aggregates displacements and accelerations into the input model, and finally predicts the output displacements.
  arXiv  Detail & Related papers  (2024-02-21T17:51:30Z)
• Evaluation of Differentially Constrained Motion Models for Graph-Based Trajectory Prediction [1.1947990549568765]
  This research investigates the performance of various motion models in combination with numerical solvers for the prediction task. The study shows that simpler models, such as low-order integrator models, are preferred over more complex, e.g., kinematic models, to achieve accurate predictions.
  arXiv  Detail & Related papers  (2023-04-11T10:15:20Z)
• Context-aware multi-head self-attentional neural network model for next location prediction [19.640761373993417]
  We utilize a multi-head self-attentional (A) neural network that learns location patterns from historical location visits. We demonstrate that proposed the model outperforms other state-of-the-art prediction models. We believe that the proposed model is vital for context-aware mobility prediction.
  arXiv  Detail & Related papers  (2022-12-04T23:40:14Z)
• PREF: Predictability Regularized Neural Motion Fields [68.60019434498703]
  Knowing 3D motions in a dynamic scene is essential to many vision applications. We leverage a neural motion field for estimating the motion of all points in a multiview setting. We propose to regularize the estimated motion to be predictable.
  arXiv  Detail & Related papers  (2022-09-21T22:32:37Z)
• Conditioned Human Trajectory Prediction using Iterative Attention Blocks [70.36888514074022]
  We present a simple yet effective pedestrian trajectory prediction model aimed at pedestrians positions prediction in urban-like environments. Our model is a neural-based architecture that can run several layers of attention blocks and transformers in an iterative sequential fashion. We show that without explicit introduction of social masks, dynamical models, social pooling layers, or complicated graph-like structures, it is possible to produce on par results with SoTA models.
  arXiv  Detail & Related papers  (2022-06-29T07:49:48Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Goal-driven Self-Attentive Recurrent Networks for Trajectory Prediction [31.02081143697431]
  Human trajectory forecasting is a key component of autonomous vehicles, social-aware robots and video-surveillance applications. We propose a lightweight attention-based recurrent backbone that acts solely on past observed positions. We employ a common goal module, based on a U-Net architecture, which additionally extracts semantic information to predict scene-compliant destinations.
  arXiv  Detail & Related papers  (2022-04-25T11:12:37Z)
• Causal-based Time Series Domain Generalization for Vehicle Intention Prediction [19.944268567657307]
  Accurately predicting possible behaviors of traffic participants is an essential capability for autonomous vehicles. In this paper, we aim to address the domain generalization problem for vehicle intention prediction tasks. Our proposed method has consistent improvement on prediction accuracy compared to other state-of-the-art domain generalization and behavior prediction methods.
  arXiv  Detail & Related papers  (2021-12-03T18:58:07Z)
• Learning Accurate Long-term Dynamics for Model-based Reinforcement Learning [7.194382512848327]
  We propose a new parametrization to supervised learning on state-action data to stably predict at longer horizons. Our results in simulated and experimental robotic tasks show that our trajectory-based models yield significantly more accurate long term predictions.
  arXiv  Detail & Related papers  (2020-12-16T18:47:37Z)
• Generative Temporal Difference Learning for Infinite-Horizon Prediction [101.59882753763888]
  We introduce the $gamma$-model, a predictive model of environment dynamics with an infinite probabilistic horizon. We discuss how its training reflects an inescapable tradeoff between training-time and testing-time compounding errors.
  arXiv  Detail & Related papers  (2020-10-27T17:54:12Z)
• Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data [37.176411554794214]
  Reasoning about human motion is an important prerequisite to safe and socially-aware robotic navigation. We present Trajectron++, a modular, graph-structured recurrent model that forecasts the trajectories of a general number of diverse agents. We demonstrate its performance on several challenging real-world trajectory forecasting datasets.
  arXiv  Detail & Related papers  (2020-01-09T16:47:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.