Learning an Interpretable Graph Structure in Multi-Task Learning

• URL: http://arxiv.org/abs/2009.05618v1
• Date: Fri, 11 Sep 2020 18:58:14 GMT
• Title: Learning an Interpretable Graph Structure in Multi-Task Learning
• Authors: Shujian Yu, Francesco Alesiani, Ammar Shaker, Wenzhe Yin
• Abstract summary: We present a novel methodology to jointly perform multi-task learning and infer intrinsic relationship among tasks by an interpretable and sparse graph. Our graph is learned simultaneously with model parameters of each task, thus it reflects the critical relationship among tasks in the specific prediction problem.
• Score: 18.293397644865454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel methodology to jointly perform multi-task learning and infer intrinsic relationship among tasks by an interpretable and sparse graph. Unlike existing multi-task learning methodologies, the graph structure is not assumed to be known a priori or estimated separately in a preprocessing step. Instead, our graph is learned simultaneously with model parameters of each task, thus it reflects the critical relationship among tasks in the specific prediction problem. We characterize graph structure with its weighted adjacency matrix and show that the overall objective can be optimized alternatively until convergence. We also show that our methodology can be simply extended to a nonlinear form by being embedded into a multi-head radial basis function network (RBFN). Extensive experiments, against six state-of-the-art methodologies, on both synthetic data and real-world applications suggest that our methodology is able to reduce generalization error, and, at the same time, reveal a sparse graph over tasks that is much easier to interpret.

Related papers

• Scalable Deep Metric Learning on Attributed Graphs [10.092560681589578]
  We develop a graph embedding method, which is based on extending deep metric and unbiased contrastive learning techniques. Based on a multi-classt loss function, we present two algorithms -- DMT for semi-supervised learning and DMAT-i for the unsupervised case.
  arXiv  Detail & Related papers  (2024-11-20T03:34:31Z)
• On Discriminative Probabilistic Modeling for Self-Supervised Representation Learning [85.75164588939185]
  We study the discriminative probabilistic modeling problem on a continuous domain for (multimodal) self-supervised representation learning. We conduct generalization error analysis to reveal the limitation of current InfoNCE-based contrastive loss for self-supervised representation learning.
  arXiv  Detail & Related papers  (2024-10-11T18:02:46Z)
• Learning Latent Graph Structures and their Uncertainty [63.95971478893842]
  Graph Neural Networks (GNNs) use relational information as an inductive bias to enhance the model's accuracy. As task-relevant relations might be unknown, graph structure learning approaches have been proposed to learn them while solving the downstream prediction task.
  arXiv  Detail & Related papers  (2024-05-30T10:49:22Z)
• Exploring Correlations of Self-Supervised Tasks for Graphs [6.977921096191354]
  This paper aims to provide a fresh understanding of graph self-supervised learning based on task correlations. We evaluate the performance of the representations trained by one specific task on other tasks and define correlation values to quantify task correlations. We propose Graph Task Correlation Modeling (GraphTCM) to illustrate the task correlations and utilize it to enhance graph self-supervised training.
  arXiv  Detail & Related papers  (2024-05-07T12:02:23Z)
• Rethinking Dimensional Rationale in Graph Contrastive Learning from Causal Perspective [15.162584339143239]
  Graph contrastive learning is a general learning paradigm excelling at capturing invariant information from diverse perturbations in graphs. Recent works focus on exploring the structural rationale from graphs, thereby increasing the discriminability of the invariant information. We propose the dimensional rationale-aware graph contrastive learning approach, which introduces a learnable dimensional rationale acquiring network and a redundancy reduction constraint.
  arXiv  Detail & Related papers  (2023-12-16T10:05:18Z)
• Bures-Wasserstein Means of Graphs [60.42414991820453]
  We propose a novel framework for defining a graph mean via embeddings in the space of smooth graph signal distributions. By finding a mean in this embedding space, we can recover a mean graph that preserves structural information. We establish the existence and uniqueness of the novel graph mean, and provide an iterative algorithm for computing it.
  arXiv  Detail & Related papers  (2023-05-31T11:04:53Z)
• Sparse Graph Learning from Spatiotemporal Time Series [16.427698929775023]
  We propose a graph learning framework that learns the relational dependencies as distributions over graphs. We show that the proposed solution can be used as a stand-alone graph identification procedure as well as a graph learning component of an end-to-end forecasting architecture.
  arXiv  Detail & Related papers  (2022-05-26T17:02:43Z)
• Bayesian Graph Contrastive Learning [55.36652660268726]
  We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders. Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector. We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-15T01:45:32Z)
• From Canonical Correlation Analysis to Self-supervised Graph Neural Networks [99.44881722969046]
  We introduce a conceptually simple yet effective model for self-supervised representation learning with graph data. We optimize an innovative feature-level objective inspired by classical Canonical Correlation Analysis. Our method performs competitively on seven public graph datasets.
  arXiv  Detail & Related papers  (2021-06-23T15:55:47Z)
• Towards Interpretable Multi-Task Learning Using Bilevel Programming [18.293397644865454]
  Interpretable Multi-Task Learning can be expressed as learning a sparse graph of the task relationship based on the prediction performance of the learned models. We show empirically how the induced sparse graph improves the interpretability of the learned models and their relationship on synthetic and real data, without sacrificing generalization performance.
  arXiv  Detail & Related papers  (2020-09-11T15:04:27Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.