Social-STAGE: Spatio-Temporal Multi-Modal Future Trajectory Forecast

• URL: http://arxiv.org/abs/2011.04853v2
• Date: Wed, 24 Mar 2021 21:32:44 GMT
• Title: Social-STAGE: Spatio-Temporal Multi-Modal Future Trajectory Forecast
• Abstract summary: We propose Social-STAGE, Social interaction-aware Spatio-Temporal multi-Attention Graph convolution network with novel Evaluation for multi-modality. Our main contributions include analysis and formulation of multi-modality with ranking using interaction and multi-attention, and introduction of new metrics to evaluate the diversity and associated confidence of multi-modal predictions.
• Score: 28.14120504421402
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper considers the problem of multi-modal future trajectory forecast with ranking. Here, multi-modality and ranking refer to the multiple plausible path predictions and the confidence in those predictions, respectively. We propose Social-STAGE, Social interaction-aware Spatio-Temporal multi-Attention Graph convolution network with novel Evaluation for multi-modality. Our main contributions include analysis and formulation of multi-modality with ranking using interaction and multi-attention, and introduction of new metrics to evaluate the diversity and associated confidence of multi-modal predictions. We evaluate our approach on existing public datasets ETH and UCY and show that the proposed algorithm outperforms the state of the arts on these datasets.

Related papers

• Benchmarking Multi-Modal Graph-based Social Media Popularity Prediction [9.450213651192836]
  We introduce MMG-Pop, a Multi-modal Graph-based Popularity Prediction benchmark.<n>We also propose MMG-PopNet, a unified multi-modal graph-based network that jointly models the aforementioned multi-modal signals and graph-structured social interactions.<n>These findings establish a unified foundation for future research on social dynamics modeling and intervention under heterogeneous modalities and socially-aware agentic ecosystem paradigms.
  arXiv  Detail & Related papers  (2026-06-25T20:39:41Z)
• Multiple-Prediction-Powered Inference [57.30769275944024]
  This work provides theoretical guarantees about the minimax optimality, finite-sample performance, and normality of the MultiPPI estimator.<n>We show that MultiPPI consistently achieves lower estimation error than existing baselines.
  arXiv  Detail & Related papers  (2026-03-28T21:24:00Z)
• Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey [124.23247710880008]
  multimodal CoT (MCoT) reasoning has recently garnered significant research attention. Existing MCoT studies design various methodologies to address the challenges of image, video, speech, audio, 3D, and structured data. We present the first systematic survey of MCoT reasoning, elucidating the relevant foundational concepts and definitions.
  arXiv  Detail & Related papers  (2025-03-16T18:39:13Z)
• Conformal Prediction for Multimodal Regression [0.0]
  conformal prediction is now extended to multimodal contexts through our methodology. Our findings highlight the potential for internal neural network features, extracted from convergence points where multimodal information is combined. This capability paves new paths for deploying conformal prediction in domains abundant with multimodal data.
  arXiv  Detail & Related papers  (2024-10-25T15:56:39Z)
• PredBench: Benchmarking Spatio-Temporal Prediction across Diverse Disciplines [86.36060279469304]
  We introduce PredBench, a benchmark tailored for the holistic evaluation of prediction-temporal networks. This benchmark integrates 12 widely adopted methods with diverse datasets across multiple application domains. Its multi-dimensional evaluation framework broadens the analysis with a comprehensive set of metrics.
  arXiv  Detail & Related papers  (2024-07-11T11:51:36Z)
• MGCP: A Multi-Grained Correlation based Prediction Network for Multivariate Time Series [54.91026286579748]
  We propose a Multi-Grained Correlations-based Prediction Network. It simultaneously considers correlations at three levels to enhance prediction performance. It employs adversarial training with an attention mechanism-based predictor and conditional discriminator to optimize prediction results at coarse-grained level.
  arXiv  Detail & Related papers  (2024-05-30T03:32:44Z)
• Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework [89.8609061423685]
  We propose an information-theoretic approach to quantify the degree of redundancy, uniqueness, and synergy relating input modalities with an output task. To validate PID estimation, we conduct extensive experiments on both synthetic datasets where the PID is known and on large-scale multimodal benchmarks. We demonstrate their usefulness in (1) quantifying interactions within multimodal datasets, (2) quantifying interactions captured by multimodal models, (3) principled approaches for model selection, and (4) three real-world case studies.
  arXiv  Detail & Related papers  (2023-02-23T18:59:05Z)
• Multimodal Trajectory Prediction: A Survey [13.519480642785561]
  Trajectory prediction is an important task to support safe and intelligent behaviours in autonomous systems. New task named multimodal trajectory prediction (MTP) aims to generate a diverse, acceptable and explainable distribution of future predictions for each agent.
  arXiv  Detail & Related papers  (2023-02-21T06:11:08Z)
• Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictions [40.70793282367128]
  We propose Multimodal Contrastive Learning for Multimodal Review Helpfulness Prediction (MRHP) problem. In addition, we introduce Adaptive Weighting scheme for our contrastive learning approach. Finally, we propose Multimodal Interaction module to address the unalignment nature of multimodal data.
  arXiv  Detail & Related papers  (2022-11-07T13:05:56Z)
• Collaborative Uncertainty Benefits Multi-Agent Multi-Modal Trajectory Forecasting [39.73793468422024]
  This work first proposes a novel concept, collaborative uncertainty (CU), which models the uncertainty resulting from interaction modules. We build a general CU-aware regression framework with an original permutation-equivariant uncertainty estimator to do both tasks of regression and uncertainty estimation. We apply the proposed framework to current SOTA multi-agent trajectory forecasting systems as a plugin module.
  arXiv  Detail & Related papers  (2022-07-11T21:17:41Z)
• Online Multi-Agent Forecasting with Interpretable Collaborative Graph Neural Network [65.11999700562869]
  We propose a novel collaborative prediction unit (CoPU), which aggregates predictions from multiple collaborative predictors according to a collaborative graph. Our methods outperform state-of-the-art works on the three tasks by 28.6%, 17.4% and 21.0% on average.
  arXiv  Detail & Related papers  (2021-07-02T08:20:06Z)
• Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesis [54.249502356251085]
  We present a novel insight along with a brand-new prediction framework. Our proposed method surpasses state-of-the-art works even without introducing social and map information.
  arXiv  Detail & Related papers  (2021-03-14T06:21:03Z)
• SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
  We propose advances that address two key challenges in future trajectory prediction. multimodality in both training data and predictions and constant time inference regardless of number of agents.
  arXiv  Detail & Related papers  (2020-07-26T08:17:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.