Causal Discovery in Physical Systems from Videos
- URL: http://arxiv.org/abs/2007.00631v3
- Date: Sun, 29 Nov 2020 20:47:06 GMT
- Title: Causal Discovery in Physical Systems from Videos
- Authors: Yunzhu Li, Antonio Torralba, Animashree Anandkumar, Dieter Fox,
Animesh Garg
- Abstract summary: Causal discovery is at the core of human cognition.
We consider the task of causal discovery from videos in an end-to-end fashion without supervision on the ground-truth graph structure.
- Score: 123.79211190669821
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Causal discovery is at the core of human cognition. It enables us to reason
about the environment and make counterfactual predictions about unseen
scenarios that can vastly differ from our previous experiences. We consider the
task of causal discovery from videos in an end-to-end fashion without
supervision on the ground-truth graph structure. In particular, our goal is to
discover the structural dependencies among environmental and object variables:
inferring the type and strength of interactions that have a causal effect on
the behavior of the dynamical system. Our model consists of (a) a perception
module that extracts a semantically meaningful and temporally consistent
keypoint representation from images, (b) an inference module for determining
the graph distribution induced by the detected keypoints, and (c) a dynamics
module that can predict the future by conditioning on the inferred graph. We
assume access to different configurations and environmental conditions, i.e.,
data from unknown interventions on the underlying system; thus, we can hope to
discover the correct underlying causal graph without explicit interventions. We
evaluate our method in a planar multi-body interaction environment and
scenarios involving fabrics of different shapes like shirts and pants.
Experiments demonstrate that our model can correctly identify the interactions
from a short sequence of images and make long-term future predictions. The
causal structure assumed by the model also allows it to make counterfactual
predictions and extrapolate to systems of unseen interaction graphs or graphs
of various sizes.
Related papers
- Learning Discrete Concepts in Latent Hierarchical Models [73.01229236386148]
Learning concepts from natural high-dimensional data holds potential in building human-aligned and interpretable machine learning models.
We formalize concepts as discrete latent causal variables that are related via a hierarchical causal model.
We substantiate our theoretical claims with synthetic data experiments.
arXiv Detail & Related papers (2024-06-01T18:01:03Z) - Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption [64.07607726562841]
Existing multi-person human reconstruction approaches mainly focus on recovering accurate poses or avoiding penetration.
In this work, we tackle the task of reconstructing closely interactive humans from a monocular video.
We propose to leverage knowledge from proxemic behavior and physics to compensate the lack of visual information.
arXiv Detail & Related papers (2024-04-17T11:55:45Z) - PGODE: Towards High-quality System Dynamics Modeling [40.76121531452706]
This paper studies the problem of modeling multi-agent dynamical systems, where agents could interact mutually to influence their behaviors.
Recent research predominantly uses geometric graphs to depict these mutual interactions, which are then captured by graph neural networks (GNNs)
We propose a new approach named Prototypical Graph ODE to address the problem.
arXiv Detail & Related papers (2023-11-11T12:04:47Z) - Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
We propose to expose captured knowledge in the form of a directed acyclic causal graph.
We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs.
The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
arXiv Detail & Related papers (2023-09-30T20:59:42Z) - Conditioned Human Trajectory Prediction using Iterative Attention Blocks [70.36888514074022]
We present a simple yet effective pedestrian trajectory prediction model aimed at pedestrians positions prediction in urban-like environments.
Our model is a neural-based architecture that can run several layers of attention blocks and transformers in an iterative sequential fashion.
We show that without explicit introduction of social masks, dynamical models, social pooling layers, or complicated graph-like structures, it is possible to produce on par results with SoTA models.
arXiv Detail & Related papers (2022-06-29T07:49:48Z) - Learning Interacting Dynamical Systems with Latent Gaussian Process ODEs [13.436770170612295]
We study for the first time uncertainty-aware modeling of continuous-time dynamics of interacting objects.
Our model infers both independent dynamics and their interactions with reliable uncertainty estimates.
arXiv Detail & Related papers (2022-05-24T08:36:25Z) - GINA: Neural Relational Inference From Independent Snapshots [0.0]
We propose a graph neural network (GNN) to simultaneously learn the latent interaction graph and, conditioned on the interaction graph, the prediction of a node's observable state.
GINA is based on the hypothesis that the ground truth interaction graph -- among all other potential graphs -- allows to predict the state of a node, given the states of its neighbors, with the highest accuracy.
arXiv Detail & Related papers (2021-05-29T15:42:33Z) - To do or not to do: finding causal relations in smart homes [2.064612766965483]
This paper introduces a new way to learn causal models from a mixture of experiments on the environment and observational data.
The core of our method is the use of selected interventions, especially our learning takes into account the variables where it is impossible to intervene.
We use our method on a smart home simulation, a use case where knowing causal relations pave the way towards explainable systems.
arXiv Detail & Related papers (2021-05-20T22:36:04Z) - Visual Grounding of Learned Physical Models [66.04898704928517]
Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions.
We present a neural model that simultaneously reasons about physics and makes future predictions based on visual and dynamics priors.
Experiments show that our model can infer the physical properties within a few observations, which allows the model to quickly adapt to unseen scenarios and make accurate predictions into the future.
arXiv Detail & Related papers (2020-04-28T17:06:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.