Navigating causal deep learning

• URL: http://arxiv.org/abs/2212.00911v1
• Date: Thu, 1 Dec 2022 23:44:23 GMT
• Title: Navigating causal deep learning
• Authors: Jeroen Berrevoets, Krzysztof Kacprzyk, Zhaozhi Qian, Mihaela van der Schaar
• Abstract summary: Causal deep learning (CDL) is a new and important research area in the larger field of machine learning. This paper categorises methods in causal deep learning beyond Pearl's ladder of causation. Our paradigm is a tool which helps researchers to: find benchmarks, compare methods, and most importantly: identify research gaps.
• Score: 78.572170629379
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal deep learning (CDL) is a new and important research area in the larger field of machine learning. With CDL, researchers aim to structure and encode causal knowledge in the extremely flexible representation space of deep learning models. Doing so will lead to more informed, robust, and general predictions and inference -- which is important! However, CDL is still in its infancy. For example, it is not clear how we ought to compare different methods as they are so different in their output, the way they encode causal knowledge, or even how they represent this knowledge. This is a living paper that categorises methods in causal deep learning beyond Pearl's ladder of causation. We refine the rungs in Pearl's ladder, while also adding a separate dimension that categorises the parametric assumptions of both input and representation, arriving at the map of causal deep learning. Our map covers machine learning disciplines such as supervised learning, reinforcement learning, generative modelling and beyond. Our paradigm is a tool which helps researchers to: find benchmarks, compare methods, and most importantly: identify research gaps. With this work we aim to structure the avalanche of papers being published on causal deep learning. While papers on the topic are being published daily, our map remains fixed. We open-source our map for others to use as they see fit: perhaps to offer guidance in a related works section, or to better highlight the contribution of their paper.

Related papers

• Towards a Categorical Foundation of Deep Learning: A Survey [0.0]
  This thesis is a survey that covers some recent work attempting to study machine learning categorically. acting as a lingua franca of mathematics and science, category theory might be able to give a unifying structure to the field of machine learning.
  arXiv  Detail & Related papers  (2024-10-07T13:11:16Z)
• Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations [1.9580473532948401]
  This thesis explores the theoretical foundations of deep learning by studying the relationship between the architecture of these models and the inherent structures found within the data they process. We ask What drives the efficacy of deep learning algorithms and allows them to beat the so-called curse of dimensionality. Our methodology takes an empirical approach to deep learning, combining experimental studies with physics-inspired toy models.
  arXiv  Detail & Related papers  (2023-10-24T19:50:41Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Adding Context to Source Code Representations for Deep Learning [13.676416860721877]
  We argue that it is beneficial for deep learning models to have access to additional contextual information about the code being analysed. We present preliminary evidence that encoding context from the call hierarchy along with information from the code itself can improve the performance of a state-of-the-art deep learning model.
  arXiv  Detail & Related papers  (2022-07-30T12:47:32Z)
• Deep Depth Completion: A Survey [26.09557446012222]
  We provide a comprehensive literature review that helps readers better grasp the research trends and clearly understand the current advances. We investigate the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies. We present a quantitative comparison of model performance on two widely used benchmark datasets, including an indoor and an outdoor dataset.
  arXiv  Detail & Related papers  (2022-05-11T08:24:00Z)
• A Reflection on Learning from Data: Epistemology Issues and Limitations [1.8047694351309205]
  This paper reflects on some issues and some limitations of the knowledge discovered in data. The paper sheds some light on the shortcomings of using generic mathematical theories to describe the process. It further highlights the need for theories specialized in learning from data.
  arXiv  Detail & Related papers  (2021-07-28T11:05:34Z)
• Ten Quick Tips for Deep Learning in Biology [116.78436313026478]
  Machine learning is concerned with the development and applications of algorithms that can recognize patterns in data and use them for predictive modeling. Deep learning has become its own subfield of machine learning. In the context of biological research, deep learning has been increasingly used to derive novel insights from high-dimensional biological data.
  arXiv  Detail & Related papers  (2021-05-29T21:02:44Z)
• Fact-driven Logical Reasoning for Machine Reading Comprehension [82.58857437343974]
  We are motivated to cover both commonsense and temporary knowledge clues hierarchically. Specifically, we propose a general formalism of knowledge units by extracting backbone constituents of the sentence. We then construct a supergraph on top of the fact units, allowing for the benefit of sentence-level (relations among fact groups) and entity-level interactions.
  arXiv  Detail & Related papers  (2021-05-21T13:11:13Z)
• Probing Across Time: What Does RoBERTa Know and When? [70.20775905353794]
  We show that linguistic knowledge is acquired fast, stably, and robustly across domains. Facts and commonsense are slower and more domain-sensitive. We believe that probing-across-time analyses can help researchers understand the complex, intermingled learning that these models undergo and guide us toward more efficient approaches that accomplish necessary learning faster.
  arXiv  Detail & Related papers  (2021-04-16T04:26:39Z)
• KILT: a Benchmark for Knowledge Intensive Language Tasks [102.33046195554886]
  We present a benchmark for knowledge-intensive language tasks (KILT) All tasks in KILT are grounded in the same snapshot of Wikipedia. We find that a shared dense vector index coupled with a seq2seq model is a strong baseline.
  arXiv  Detail & Related papers  (2020-09-04T15:32:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.