Invariant Structure Learning for Better Generalization and Causal
Explainability
- URL: http://arxiv.org/abs/2206.06469v1
- Date: Mon, 13 Jun 2022 21:04:23 GMT
- Title: Invariant Structure Learning for Better Generalization and Causal
Explainability
- Authors: Yunhao Ge, Sercan \"O. Arik, Jinsung Yoon, Ao Xu, Laurent Itti, Tomas
Pfister
- Abstract summary: We propose a novel framework, Invariant Structure Learning (ISL), to improve causal structure discovery.
ISL splits the data into different environments, and learns a structure that is invariant to the target across different environments.
We demonstrate that ISL accurately discovers the causal structure, outperforms alternative methods, and yields superior generalization for datasets with significant distribution shifts.
- Score: 44.580704853704994
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Learning the causal structure behind data is invaluable for improving
generalization and obtaining high-quality explanations. We propose a novel
framework, Invariant Structure Learning (ISL), that is designed to improve
causal structure discovery by utilizing generalization as an indication. ISL
splits the data into different environments, and learns a structure that is
invariant to the target across different environments by imposing a consistency
constraint. An aggregation mechanism then selects the optimal classifier based
on a graph structure that reflects the causal mechanisms in the data more
accurately compared to the structures learnt from individual environments.
Furthermore, we extend ISL to a self-supervised learning setting where accurate
causal structure discovery does not rely on any labels. This self-supervised
ISL utilizes invariant causality proposals by iteratively setting different
nodes as targets. On synthetic and real-world datasets, we demonstrate that ISL
accurately discovers the causal structure, outperforms alternative methods, and
yields superior generalization for datasets with significant distribution
shifts.
Related papers
- Learning Invariant Representations of Graph Neural Networks via Cluster
Generalization [58.68231635082891]
Graph neural networks (GNNs) have become increasingly popular in modeling graph-structured data.
In this paper, we experimentally find that the performance of GNNs drops significantly when the structure shift happens.
We propose the Cluster Information Transfer (CIT) mechanism, which can learn invariant representations for GNNs.
arXiv Detail & Related papers (2024-03-06T10:36:56Z) - Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Network [29.149367323751413]
We propose ReStruct, a meta-structure search framework that integrates reasoning into the evolutionary procedure.
We show that ReStruct achieves state-of-the-art performance in both recommendation and node classification tasks.
arXiv Detail & Related papers (2024-02-18T09:21:12Z) - Structured Language Generation Model for Robust Structure Prediction [6.4736137270915215]
We propose a framework that reduces sequence-to-sequence problems to classification problems via methodologies in loss calibration and decoding method.
Our experimental results show that SLGM is able to maintain performance without explicit dataset information, follow and potentially replace dataset-specific fine-tuning.
arXiv Detail & Related papers (2024-02-14T06:33:22Z) - iSCAN: Identifying Causal Mechanism Shifts among Nonlinear Additive
Noise Models [48.33685559041322]
This paper focuses on identifying the causal mechanism shifts in two or more related datasets over the same set of variables.
Code implementing the proposed method is open-source and publicly available at https://github.com/kevinsbello/iSCAN.
arXiv Detail & Related papers (2023-06-30T01:48:11Z) - SE-GSL: A General and Effective Graph Structure Learning Framework
through Structural Entropy Optimization [67.28453445927825]
Graph Neural Networks (GNNs) are de facto solutions to structural data learning.
Existing graph structure learning (GSL) frameworks still lack robustness and interpretability.
This paper proposes a general GSL framework, SE-GSL, through structural entropy and the graph hierarchy abstracted in the encoding tree.
arXiv Detail & Related papers (2023-03-17T05:20:24Z) - Amortized Inference for Causal Structure Learning [72.84105256353801]
Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test.
We train a variational inference model to predict the causal structure from observational/interventional data.
Our models exhibit robust generalization capabilities under substantial distribution shift.
arXiv Detail & Related papers (2022-05-25T17:37:08Z) - GenURL: A General Framework for Unsupervised Representation Learning [58.59752389815001]
Unsupervised representation learning (URL) learns compact embeddings of high-dimensional data without supervision.
We propose a unified similarity-based URL framework, GenURL, which can smoothly adapt to various URL tasks.
Experiments demonstrate that GenURL achieves consistent state-of-the-art performance in self-supervised visual learning, unsupervised knowledge distillation (KD), graph embeddings (GE), and dimension reduction.
arXiv Detail & Related papers (2021-10-27T16:24:39Z) - Structure-Aware Feature Generation for Zero-Shot Learning [108.76968151682621]
We introduce a novel structure-aware feature generation scheme, termed as SA-GAN, to account for the topological structure in learning both the latent space and the generative networks.
Our method significantly enhances the generalization capability on unseen-classes and consequently improve the classification performance.
arXiv Detail & Related papers (2021-08-16T11:52:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.