Unifying Causal Inference and Reinforcement Learning using Higher-Order Category Theory

• URL: http://arxiv.org/abs/2209.06262v1
• Date: Tue, 13 Sep 2022 19:04:18 GMT
• Title: Unifying Causal Inference and Reinforcement Learning using Higher-Order Category Theory
• Authors: Sridhar Mahadevan
• Abstract summary: We present a unified formalism for structure discovery of causal models and predictive state representation models in reinforcement learning. Specifically, we model structure discovery in both settings using simplicial objects.
• Score: 4.119151469153588
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a unified formalism for structure discovery of causal models and predictive state representation (PSR) models in reinforcement learning (RL) using higher-order category theory. Specifically, we model structure discovery in both settings using simplicial objects, contravariant functors from the category of ordinal numbers into any category. Fragments of causal models that are equivalent under conditional independence -- defined as causal horns -- as well as subsequences of potential tests in a predictive state representation -- defined as predictive horns -- are both special cases of horns of a simplicial object, subsets resulting from the removal of the interior and the face opposite a particular vertex. Latent structure discovery in both settings involve the same fundamental mathematical problem of finding extensions of horns of simplicial objects through solving lifting problems in commutative diagrams, and exploiting weak homotopies that define higher-order symmetries. Solutions to the problem of filling "inner" vs "outer" horns leads to various notions of higher-order categories, including weak Kan complexes and quasicategories. We define the abstract problem of structure discovery in both settings in terms of adjoint functors between the category of universal causal models or universal decision models and its simplicial object representation.

Related papers

• What makes Models Compositional? A Theoretical View: With Supplement [60.284698521569936]
  We propose a general neuro-symbolic definition of compositional functions and their compositional complexity. We show how various existing general and special purpose sequence processing models fit this definition and use it to analyze their compositional complexity.
  arXiv  Detail & Related papers  (2024-05-02T20:10:27Z)
• Algebraic Models for Qualified Aggregation in General Rough Sets, and Reasoning Bias Discovery [0.0]
  The research is motivated by the desire to model skeptical or pessimistic, and optimistic or possibilistic aggregation in human reasoning. The model is suitable for the study of discriminatory/toxic behavior in human reasoning, and of ML algorithms learning such behavior.
  arXiv  Detail & Related papers  (2023-08-30T17:07:54Z)
• Causal models in string diagrams [0.0]
  The framework of causal models provides a principled approach to causal reasoning, applied today across many scientific domains. We present this framework in the language of string diagrams, interpreted formally using category theory. We argue and demonstrate that causal reasoning according to the causal model framework is most naturally and intuitively done as diagrammatic reasoning.
  arXiv  Detail & Related papers  (2023-04-15T21:54:48Z)
• Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations [62.65877150123775]
  Causal abstraction is a promising theoretical framework for explainable artificial intelligence. Existing causal abstraction methods require a brute-force search over alignments between the high-level model and the low-level one. We present distributed alignment search (DAS), which overcomes these limitations.
  arXiv  Detail & Related papers  (2023-03-05T00:57:49Z)
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion [66.21290235237808]
  We introduce an energy constrained diffusion model which encodes a batch of instances from a dataset into evolutionary states. We provide rigorous theory that implies closed-form optimal estimates for the pairwise diffusion strength among arbitrary instance pairs. Experiments highlight the wide applicability of our model as a general-purpose encoder backbone with superior performance in various tasks.
  arXiv  Detail & Related papers  (2023-01-23T15:18:54Z)
• Invertible bimodule categories and generalized Schur orthogonality [0.0]
  We use a generalization to weak Hopf algebras to determine whether a given bimodule category is invertible. We show that our condition for invertibility is equivalent to the notion of MPO-injectivity.
  arXiv  Detail & Related papers  (2022-11-03T16:34:01Z)
• On The Universality of Diagrams for Causal Inference and The Causal Reproducing Property [4.119151469153588]
  We propose a framework based on category theory that defines the universal property that underlies causal inference. We present two foundational results in universal causal inference.
  arXiv  Detail & Related papers  (2022-07-06T18:54:15Z)
• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning [76.00395335702572]
  A central goal for AI and causality is the joint discovery of abstract representations and causal structure. Existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs. In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them.
  arXiv  Detail & Related papers  (2021-07-02T05:44:56Z)
• Models of zero-range interaction for the bosonic trimer at unitarity [91.3755431537592]
  We present the construction of quantum Hamiltonians for a three-body system consisting of identical bosons mutually coupled by a two-body interaction of zero range. For a large part of the presentation, infinite scattering length will be considered.
  arXiv  Detail & Related papers  (2020-06-03T17:54:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.