Differentiable and Transportable Structure Learning

• URL: http://arxiv.org/abs/2206.06354v4
• Date: Mon, 12 Jun 2023 10:22:27 GMT
• Title: Differentiable and Transportable Structure Learning
• Authors: Jeroen Berrevoets, Nabeel Seedat, Fergus Imrie, Mihaela van der Schaar
• Abstract summary: We introduce D-Struct, which recovers transportability in the discovered structures through a novel architecture and loss function. Because D-Struct remains differentiable, our method can be easily adopted in existing differentiable architectures.
• Score: 73.84540901950616
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Directed acyclic graphs (DAGs) encode a lot of information about a particular distribution in their structure. However, compute required to infer these structures is typically super-exponential in the number of variables, as inference requires a sweep of a combinatorially large space of potential structures. That is, until recent advances made it possible to search this space using a differentiable metric, drastically reducing search time. While this technique -- named NOTEARS -- is widely considered a seminal work in DAG-discovery, it concedes an important property in favour of differentiability: transportability. To be transportable, the structures discovered on one dataset must apply to another dataset from the same domain. We introduce D-Struct which recovers transportability in the discovered structures through a novel architecture and loss function while remaining fully differentiable. Because D-Struct remains differentiable, our method can be easily adopted in existing differentiable architectures, as was previously done with NOTEARS. In our experiments, we empirically validate D-Struct with respect to edge accuracy and structural Hamming distance in a variety of settings.

Related papers

• Learning Discrete Latent Variable Structures with Tensor Rank Conditions [30.292492090200984]
  Unobserved discrete data are ubiquitous in many scientific disciplines, and how to learn the causal structure of these latent variables is crucial for uncovering data patterns. Most studies focus on the linear latent variable model or impose strict constraints on latent structures, which fail to address cases in discrete data involving non-linear relationships or complex latent structures. We explore a tensor rank condition on contingency tables for an observed variable set $mathbfX_p$, showing that the rank is determined by the minimum support of a specific conditional set. One can locate the latent variable through probing the rank on different observed variables
  arXiv  Detail & Related papers  (2024-06-11T07:25:17Z)
• Mitigating Noisy Correspondence by Geometrical Structure Consistency Learning [43.75697355156703]
  Noisy correspondence is prevalent on human-annotated or web-crawled datasets. We introduce a Geometrical Structure Consistency (GSC) method to infer the true correspondence.
  arXiv  Detail & Related papers  (2024-05-27T09:42:52Z)
• Coordinated Multi-Neighborhood Learning on a Directed Acyclic Graph [6.727984016678534]
  Learning the structure of causal directed acyclic graphs (DAGs) is useful in many areas of machine learning and artificial intelligence. It is challenging to obtain good empirical and theoretical results without strong and often restrictive assumptions. This paper develops a new constraint-based method for estimating the local structure around multiple user-specified target nodes.
  arXiv  Detail & Related papers  (2024-05-24T08:49:43Z)
• Neural Structure Learning with Stochastic Differential Equations [9.076396370870423]
  We introduce a novel structure learning method, SCOTCH, which combines neural differential equations with variational inference to infer a posterior distribution over possible structures. This continuous-time approach can naturally handle both learning from and predicting observations at arbitrary time points.
  arXiv  Detail & Related papers  (2023-11-06T17:58:47Z)
• How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding [56.222097640468306]
  We provide mechanistic understanding of how transformers learn "semantic structure" We show, through a combination of mathematical analysis and experiments on Wikipedia data, that the embedding layer and the self-attention layer encode the topical structure.
  arXiv  Detail & Related papers  (2023-03-07T21:42:17Z)
• DA-VEGAN: Differentiably Augmenting VAE-GAN for microstructure reconstruction from extremely small data sets [110.60233593474796]
  DA-VEGAN is a model with two central innovations. A $beta$-variational autoencoder is incorporated into a hybrid GAN architecture. A custom differentiable data augmentation scheme is developed specifically for this architecture.
  arXiv  Detail & Related papers  (2023-02-17T08:49:09Z)
• Latent Hierarchical Causal Structure Discovery with Rank Constraints [19.61598654735681]
  We consider a challenging scenario for causal structure identification, where some variables are latent and they form a hierarchical graph structure. We propose an estimation procedure that can efficiently locate latent variables, determine their cardinalities, and identify the latent hierarchical structure.
  arXiv  Detail & Related papers  (2022-10-01T03:27:54Z)
• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• Structurally Diverse Sampling Reduces Spurious Correlations in Semantic Parsing Datasets [51.095144091781734]
  We propose a novel algorithm for sampling a structurally diverse set of instances from a labeled instance pool with structured outputs. We show that our algorithm performs competitively with or better than prior algorithms in not only compositional template splits but also traditional IID splits. In general, we find that diverse train sets lead to better generalization than random training sets of the same size in 9 out of 10 dataset-split pairs.
  arXiv  Detail & Related papers  (2022-03-16T07:41:27Z)
• Measuring dissimilarity with diffeomorphism invariance [94.02751799024684]
  We introduce DID, a pairwise dissimilarity measure applicable to a wide range of data spaces. We prove that DID enjoys properties which make it relevant for theoretical study and practical use.
  arXiv  Detail & Related papers  (2022-02-11T13:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.