Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective

• URL: http://arxiv.org/abs/2111.14820v1
• Date: Mon, 29 Nov 2021 18:59:09 GMT
• Title: Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective
• Authors: Yuejiang Liu, Riccardo Cadei, Jonas Schweizer, Sherwin Bahmani, Alexandre Alahi
• Abstract summary: We introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables. We devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph. Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations.
• Score: 72.55093886515824
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning behavioral patterns from observational data has been a de-facto approach to motion forecasting. Yet, the current paradigm suffers from two shortcomings: brittle under covariate shift and inefficient for knowledge transfer. In this work, we propose to address these challenges from a causal representation perspective. We first introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables, namely invariant mechanisms, style confounders, and spurious features. We then introduce a learning framework that treats each group separately: (i) unlike the common practice of merging datasets collected from different locations, we exploit their subtle distinctions by means of an invariance loss encouraging the model to suppress spurious correlations; (ii) we devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph; (iii) we introduce a style consistency loss that not only enforces the structure of style representations but also serves as a self-supervisory signal for test-time refinement on the fly. Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations, outperforming prior state-of-the-art motion forecasting models for out-of-distribution generalization and low-shot transfer.

Related papers

• Sequential Representation Learning via Static-Dynamic Conditional Disentanglement [58.19137637859017]
  This paper explores self-supervised disentangled representation learning within sequential data, focusing on separating time-independent and time-varying factors in videos. We propose a new model that breaks the usual independence assumption between those factors by explicitly accounting for the causal relationship between the static/dynamic variables. Experiments show that the proposed approach outperforms previous complex state-of-the-art techniques in scenarios where the dynamics of a scene are influenced by its content.
  arXiv  Detail & Related papers  (2024-08-10T17:04:39Z)
• FiP: a Fixed-Point Approach for Causal Generative Modeling [20.88890689294816]
  We propose a new and equivalent formalism that does not require DAGs to describe fixed-point problems on the causally ordered variables. We show three important cases where they can be uniquely recovered given the topological ordering (TO)
  arXiv  Detail & Related papers  (2024-04-10T12:29:05Z)
• Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
  We propose to expose captured knowledge in the form of a directed acyclic causal graph. We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs. The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-09-30T20:59:42Z)
• On the Learning Dynamics of Attention Networks [0.7614628596146599]
  Attention models are learned by optimizing one of three standard loss functions that are variously called -- soft attention, hard attention, and latent variable marginal likelihood (LVML) attention. We observe a unique signature of models learned using these paradigms and explain this as a consequence of the evolution of the classification model under gradient descent. We propose a simple hybrid approach that combines the advantages of the different loss functions and demonstrates it on a collection of semi-synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-07-25T11:40:47Z)
• Consistent Explanations in the Face of Model Indeterminacy via Ensembling [12.661530681518899]
  This work addresses the challenge of providing consistent explanations for predictive models in the presence of model indeterminacy. We introduce ensemble methods to enhance the consistency of the explanations provided in these scenarios. Our findings highlight the importance of considering model indeterminacy when interpreting explanations.
  arXiv  Detail & Related papers  (2023-06-09T18:45:43Z)
• Motion-Scenario Decoupling for Rat-Aware Video Position Prediction: Strategy and Benchmark [49.58762201363483]
  We introduce RatPose, a bio-robot motion prediction dataset constructed by considering the influence factors of individuals and environments. We propose a Dual-stream Motion-Scenario Decoupling framework that effectively separates scenario-oriented and motion-oriented features. We demonstrate significant performance improvements of the proposed textitDMSD framework on different difficulty-level tasks.
  arXiv  Detail & Related papers  (2023-05-17T14:14:31Z)
• Diffusion Action Segmentation [63.061058214427085]
  We propose a novel framework via denoising diffusion models, which shares the same inherent spirit of such iterative refinement. In this framework, action predictions are iteratively generated from random noise with input video features as conditions.
  arXiv  Detail & Related papers  (2023-03-31T10:53:24Z)
• Identifying Weight-Variant Latent Causal Models [82.14087963690561]
  We find that transitivity acts as a key role in impeding the identifiability of latent causal representations. Under some mild assumptions, we can show that the latent causal representations can be identified up to trivial permutation and scaling. We propose a novel method, termed Structural caUsAl Variational autoEncoder, which directly learns latent causal representations and causal relationships among them.
  arXiv  Detail & Related papers  (2022-08-30T11:12:59Z)
• Generalizable Information Theoretic Causal Representation [37.54158138447033]
  We propose to learn causal representation from observational data by regularizing the learning procedure with mutual information measures according to our hypothetical causal graph. The optimization involves a counterfactual loss, based on which we deduce a theoretical guarantee that the causality-inspired learning is with reduced sample complexity and better generalization ability.
  arXiv  Detail & Related papers  (2022-02-17T00:38:35Z)
• Disentangling Action Sequences: Discovering Correlated Samples [6.179793031975444]
  We demonstrate the data itself plays a crucial role in disentanglement and instead of the factors, and the disentangled representations align the latent variables with the action sequences. We propose a novel framework, fractional variational autoencoder (FVAE) to disentangle the action sequences with different significance step-by-step. Experimental results on dSprites and 3D Chairs show that FVAE improves the stability of disentanglement.
  arXiv  Detail & Related papers  (2020-10-17T07:37:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.