Adaptive Human Trajectory Prediction via Latent Corridors

• URL: http://arxiv.org/abs/2312.06653v2
• Date: Fri, 12 Jul 2024 03:30:42 GMT
• Title: Adaptive Human Trajectory Prediction via Latent Corridors
• Authors: Neerja Thakkar, Karttikeya Mangalam, Andrea Bajcsy, Jitendra Malik,
• Abstract summary: We formalize the problem of scene-specific adaptive trajectory prediction. We propose a new adaptation approach inspired by prompt tuning called latent corridors. With less than 0.1% additional model parameters, we see up to 23.9% ADE improvement in MOT Synth simulated data and 16.4% ADE in MOT and Wildtrack real pedestrian data.
• Score: 49.13061580045407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human trajectory prediction is typically posed as a zero-shot generalization problem: a predictor is learnt on a dataset of human motion in training scenes, and then deployed on unseen test scenes. While this paradigm has yielded tremendous progress, it fundamentally assumes that trends in human behavior within the deployment scene are constant over time. As such, current prediction models are unable to adapt to scene-specific transient human behaviors, such as crowds temporarily gathering to see buskers, pedestrians hurrying through the rain and avoiding puddles, or a protest breaking out. We formalize the problem of scene-specific adaptive trajectory prediction and propose a new adaptation approach inspired by prompt tuning called latent corridors. By augmenting the input of any pre-trained human trajectory predictor with learnable image prompts, the predictor can improve in the deployment scene by inferring trends from extremely small amounts of new data (e.g., 2 humans observed for 30 seconds). With less than 0.1% additional model parameters, we see up to 23.9% ADE improvement in MOTSynth simulated data and 16.4% ADE in MOT and Wildtrack real pedestrian data. Qualitatively, we observe that latent corridors imbue predictors with an awareness of scene geometry and scene-specific human behaviors that non-adaptive predictors struggle to capture. The project website can be found at https://neerja.me/atp_latent_corridors/.

Related papers

• Evaluating Human Trajectory Prediction with Metamorphic Testing [15.836913530330786]
  The prediction of human trajectories is important for planning in autonomous systems that act in the real world. No prediction does precisely match any future trajectory. We explore the application of metamorphic testing for human trajectory prediction.
  arXiv  Detail & Related papers  (2024-07-26T14:10:14Z)
• Humanoid Locomotion as Next Token Prediction [84.21335675130021]
  Our model is a causal transformer trained via autoregressive prediction of sensorimotor trajectories. We show that our model enables a full-sized humanoid to walk in San Francisco zero-shot. Our model can transfer to the real world even when trained on only 27 hours of walking data, and can generalize commands not seen during training like walking backward.
  arXiv  Detail & Related papers  (2024-02-29T18:57:37Z)
• Social-Transmotion: Promptable Human Trajectory Prediction [65.80068316170613]
  Social-Transmotion is a generic Transformer-based model that exploits diverse and numerous visual cues to predict human behavior. Our approach is validated on multiple datasets, including JTA, JRDB, Pedestrians and Cyclists in Road Traffic, and ETH-UCY.
  arXiv  Detail & Related papers  (2023-12-26T18:56:49Z)
• Robots That Can See: Leveraging Human Pose for Trajectory Prediction [30.919756497223343]
  We present a Transformer based architecture to predict human future trajectories in human-centric environments. The resulting model captures the inherent uncertainty for future human trajectory prediction. We identify new agents with limited historical data as a major contributor to error and demonstrate the complementary nature of 3D skeletal poses in reducing prediction error.
  arXiv  Detail & Related papers  (2023-09-29T13:02:56Z)
• Staged Contact-Aware Global Human Motion Forecasting [7.930326095134298]
  Scene-aware global human motion forecasting is critical for manifold applications, including virtual reality, robotics, and sports. We propose a STAGed contact-aware global human motion forecasting STAG, a novel three-stage pipeline for predicting global human motion in a 3D environment. STAG achieves a 1.8% and 16.2% overall improvement in pose and trajectory prediction, respectively, on the scene-aware GTA-IM dataset.
  arXiv  Detail & Related papers  (2023-09-16T10:47:48Z)
• Forecasting Human Trajectory from Scene History [51.72069374835107]
  We propose to forecast a person's future trajectory by learning from the implicit scene regularities. We categorize scene history information into two types: historical group trajectory and individual-surroundings interaction. We propose a novel framework Scene History Excavating Network (SHENet), where the scene history is leveraged in a simple yet effective approach.
  arXiv  Detail & Related papers  (2022-10-17T03:56:02Z)
• Towards Robust Human Trajectory Prediction in Raw Videos [8.301214274565819]
  We study the problem of human trajectory forecasting in raw videos. We show that the prediction accuracy can be severely affected by various types of tracking errors. We propose a simple yet effective strategy to correct the tracking failures by enforcing prediction consistency over time.
  arXiv  Detail & Related papers  (2021-08-18T17:27:26Z)
• SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction [64.16212996247943]
  We present a Sparse Graph Convolution Network(SGCN) for pedestrian trajectory prediction. Specifically, the SGCN explicitly models the sparse directed interaction with a sparse directed spatial graph to capture adaptive interaction pedestrians. visualizations indicate that our method can capture adaptive interactions between pedestrians and their effective motion tendencies.
  arXiv  Detail & Related papers  (2021-04-04T03:17:42Z)
• Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting [91.69900691029908]
  We advocate for predicting both the individual motions as well as the scene occupancy map. We propose a Scene-Actor Graph Neural Network (SA-GNN) which preserves the relative spatial information of pedestrians. On two large-scale real-world datasets, we showcase that our scene-occupancy predictions are more accurate and better calibrated than those from state-of-the-art motion forecasting methods.
  arXiv  Detail & Related papers  (2021-01-07T06:08:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.