Towards Robust and Adaptive Motion Forecasting: A Causal Representation
Perspective
- URL: http://arxiv.org/abs/2111.14820v1
- Date: Mon, 29 Nov 2021 18:59:09 GMT
- Title: Towards Robust and Adaptive Motion Forecasting: A Causal Representation
Perspective
- Authors: Yuejiang Liu, Riccardo Cadei, Jonas Schweizer, Sherwin Bahmani,
Alexandre Alahi
- Abstract summary: We introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables.
We devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph.
Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations.
- Score: 72.55093886515824
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Learning behavioral patterns from observational data has been a de-facto
approach to motion forecasting. Yet, the current paradigm suffers from two
shortcomings: brittle under covariate shift and inefficient for knowledge
transfer. In this work, we propose to address these challenges from a causal
representation perspective. We first introduce a causal formalism of motion
forecasting, which casts the problem as a dynamic process with three groups of
latent variables, namely invariant mechanisms, style confounders, and spurious
features. We then introduce a learning framework that treats each group
separately: (i) unlike the common practice of merging datasets collected from
different locations, we exploit their subtle distinctions by means of an
invariance loss encouraging the model to suppress spurious correlations; (ii)
we devise a modular architecture that factorizes the representations of
invariant mechanisms and style confounders to approximate a causal graph; (iii)
we introduce a style consistency loss that not only enforces the structure of
style representations but also serves as a self-supervisory signal for
test-time refinement on the fly. Experiment results on synthetic and real
datasets show that our three proposed components significantly improve the
robustness and reusability of the learned motion representations, outperforming
prior state-of-the-art motion forecasting models for out-of-distribution
generalization and low-shot transfer.
Related papers
- Neural Persistence Dynamics [8.197801260302642]
We consider the problem of learning the dynamics in the topology of time-evolving point clouds.
Our proposed model -- staticitneural persistence dynamics -- substantially outperforms the state-of-the-art across a diverse set of parameter regression tasks.
arXiv Detail & Related papers (2024-05-24T17:20:18Z) - Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
We propose to expose captured knowledge in the form of a directed acyclic causal graph.
We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs.
The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
arXiv Detail & Related papers (2023-09-30T20:59:42Z) - On the Learning Dynamics of Attention Networks [0.7614628596146599]
Attention models are learned by optimizing one of three standard loss functions that are variously called -- soft attention, hard attention, and latent variable marginal likelihood (LVML) attention.
We observe a unique signature of models learned using these paradigms and explain this as a consequence of the evolution of the classification model under gradient descent.
We propose a simple hybrid approach that combines the advantages of the different loss functions and demonstrates it on a collection of semi-synthetic and real-world datasets.
arXiv Detail & Related papers (2023-07-25T11:40:47Z) - Consistent Explanations in the Face of Model Indeterminacy via
Ensembling [12.661530681518899]
This work addresses the challenge of providing consistent explanations for predictive models in the presence of model indeterminacy.
We introduce ensemble methods to enhance the consistency of the explanations provided in these scenarios.
Our findings highlight the importance of considering model indeterminacy when interpreting explanations.
arXiv Detail & Related papers (2023-06-09T18:45:43Z) - Motion-Scenario Decoupling for Rat-Aware Video Position Prediction:
Strategy and Benchmark [49.58762201363483]
We introduce RatPose, a bio-robot motion prediction dataset constructed by considering the influence factors of individuals and environments.
We propose a Dual-stream Motion-Scenario Decoupling framework that effectively separates scenario-oriented and motion-oriented features.
We demonstrate significant performance improvements of the proposed textitDMSD framework on different difficulty-level tasks.
arXiv Detail & Related papers (2023-05-17T14:14:31Z) - Diffusion Action Segmentation [63.061058214427085]
We propose a novel framework via denoising diffusion models, which shares the same inherent spirit of such iterative refinement.
In this framework, action predictions are iteratively generated from random noise with input video features as conditions.
arXiv Detail & Related papers (2023-03-31T10:53:24Z) - Understanding and Constructing Latent Modality Structures in Multi-modal
Representation Learning [53.68371566336254]
We argue that the key to better performance lies in meaningful latent modality structures instead of perfect modality alignment.
Specifically, we design 1) a deep feature separation loss for intra-modality regularization; 2) a Brownian-bridge loss for inter-modality regularization; and 3) a geometric consistency loss for both intra- and inter-modality regularization.
arXiv Detail & Related papers (2023-03-10T14:38:49Z) - Generalizable Information Theoretic Causal Representation [37.54158138447033]
We propose to learn causal representation from observational data by regularizing the learning procedure with mutual information measures according to our hypothetical causal graph.
The optimization involves a counterfactual loss, based on which we deduce a theoretical guarantee that the causality-inspired learning is with reduced sample complexity and better generalization ability.
arXiv Detail & Related papers (2022-02-17T00:38:35Z) - Paired Examples as Indirect Supervision in Latent Decision Models [109.76417071249945]
We introduce a way to leverage paired examples that provide stronger cues for learning latent decisions.
We apply our method to improve compositional question answering using neural module networks on the DROP dataset.
arXiv Detail & Related papers (2021-04-05T03:58:30Z) - Disentangling Action Sequences: Discovering Correlated Samples [6.179793031975444]
We demonstrate the data itself plays a crucial role in disentanglement and instead of the factors, and the disentangled representations align the latent variables with the action sequences.
We propose a novel framework, fractional variational autoencoder (FVAE) to disentangle the action sequences with different significance step-by-step.
Experimental results on dSprites and 3D Chairs show that FVAE improves the stability of disentanglement.
arXiv Detail & Related papers (2020-10-17T07:37:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.