Unified Human Localization and Trajectory Prediction with Monocular Vision

• URL: http://arxiv.org/abs/2503.03535v1
• Date: Wed, 05 Mar 2025 14:18:39 GMT
• Title: Unified Human Localization and Trajectory Prediction with Monocular Vision
• Authors: Po-Chien Luan, Yang Gao, Celine Demonsant, Alexandre Alahi,
• Abstract summary: MonoTransmotion is a Transformer-based framework that uses only a monocular camera to jointly solve localization and prediction tasks.<n>We show that by jointly training both tasks with our unified framework, our method is more robust in real-world scenarios made of noisy inputs.
• Score: 64.19384064365431
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conventional human trajectory prediction models rely on clean curated data, requiring specialized equipment or manual labeling, which is often impractical for robotic applications. The existing predictors tend to overfit to clean observation affecting their robustness when used with noisy inputs. In this work, we propose MonoTransmotion (MT), a Transformer-based framework that uses only a monocular camera to jointly solve localization and prediction tasks. Our framework has two main modules: Bird's Eye View (BEV) localization and trajectory prediction. The BEV localization module estimates the position of a person using 2D human poses, enhanced by a novel directional loss for smoother sequential localizations. The trajectory prediction module predicts future motion from these estimates. We show that by jointly training both tasks with our unified framework, our method is more robust in real-world scenarios made of noisy inputs. We validate our MT network on both curated and non-curated datasets. On the curated dataset, MT achieves around 12% improvement over baseline models on BEV localization and trajectory prediction. On real-world non-curated dataset, experimental results indicate that MT maintains similar performance levels, highlighting its robustness and generalization capability. The code is available at https://github.com/vita-epfl/MonoTransmotion.

Related papers

• BEVDiffLoc: End-to-End LiDAR Global Localization in BEV View based on Diffusion Model [8.720833232645155]
  Bird's-Eye-View (BEV) image is one of the most widely adopted data representations in autonomous driving. We propose BEVDiffLoc, a novel framework that formulates LiDAR localization as a conditional generation of poses.
  arXiv  Detail & Related papers  (2025-03-14T13:17:43Z)
• ASTRA: A Scene-aware TRAnsformer-based model for trajectory prediction [15.624698974735654]
  ASTRA (A Scene-aware TRAnsformer-based model for trajectory prediction) is a light-weight pedestrian trajectory forecasting model.<n>We utilise a U-Net-based feature extractor, via its latent vector representation, to capture scene representations and a graph-aware transformer encoder for capturing social interactions.
  arXiv  Detail & Related papers  (2025-01-16T23:28:30Z)
• Multi-Transmotion: Pre-trained Model for Human Motion Prediction [68.87010221355223]
  Multi-Transmotion is an innovative transformer-based model designed for cross-modality pre-training. Our methodology demonstrates competitive performance across various datasets on several downstream tasks.
  arXiv  Detail & Related papers  (2024-11-04T23:15:21Z)
• OPUS: Occupancy Prediction Using a Sparse Set [64.60854562502523]
  We present a framework to simultaneously predict occupied locations and classes using a set of learnable queries. OPUS incorporates a suite of non-trivial strategies to enhance model performance. Our lightest model achieves superior RayIoU on the Occ3D-nuScenes dataset at near 2x FPS, while our heaviest model surpasses previous best results by 6.1 RayIoU.
  arXiv  Detail & Related papers  (2024-09-14T07:44:22Z)
• VisionTrap: Vision-Augmented Trajectory Prediction Guided by Textual Descriptions [10.748597086208145]
  In this work, we propose a novel method that also incorporates visual input from surround-view cameras. Our method achieves a latency of 53 ms, making it feasible for real-time processing. Our experiments show that both the visual inputs and the textual descriptions contribute to improvements in trajectory prediction performance.
  arXiv  Detail & Related papers  (2024-07-17T06:39:52Z)
• Vectorized Representation Dreamer (VRD): Dreaming-Assisted Multi-Agent Motion-Forecasting [2.2020053359163305]
  We introduce VRD, a vectorized world model-inspired approach to the multi-agent motion forecasting problem. Our method combines a traditional open-loop training regime with a novel dreamed closed-loop training pipeline. Our model achieves state-of-the-art performance on the single prediction miss rate metric.
  arXiv  Detail & Related papers  (2024-06-20T15:34:17Z)
• LOPR: Latent Occupancy PRediction using Generative Models [49.15687400958916]
  LiDAR generated occupancy grid maps (L-OGMs) offer a robust bird's eye-view scene representation. We propose a framework that decouples occupancy prediction into: representation learning and prediction within the learned latent space.
  arXiv  Detail & Related papers  (2022-10-03T22:04:00Z)
• Transforming Model Prediction for Tracking [109.08417327309937]
  Transformers capture global relations with little inductive bias, allowing it to learn the prediction of more powerful target models. We train the proposed tracker end-to-end and validate its performance by conducting comprehensive experiments on multiple tracking datasets. Our tracker sets a new state of the art on three benchmarks, achieving an AUC of 68.5% on the challenging LaSOT dataset.
  arXiv  Detail & Related papers  (2022-03-21T17:59:40Z)
• Injecting Knowledge in Data-driven Vehicle Trajectory Predictors [82.91398970736391]
  Vehicle trajectory prediction tasks have been commonly tackled from two perspectives: knowledge-driven or data-driven. In this paper, we propose to learn a "Realistic Residual Block" (RRB) which effectively connects these two perspectives. Our proposed method outputs realistic predictions by confining the residual range and taking into account its uncertainty.
  arXiv  Detail & Related papers  (2021-03-08T16:03:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.