Boosting Multitask Learning on Graphs through Higher-Order Task Affinities

• URL: http://arxiv.org/abs/2306.14009v4
• Date: Thu, 14 Mar 2024 22:54:18 GMT
• Title: Boosting Multitask Learning on Graphs through Higher-Order Task Affinities
• Authors: Dongyue Li, Haotian Ju, Aneesh Sharma, Hongyang R. Zhang,
• Abstract summary: Predicting node labels on a given graph is a widely studied problem with many applications, including community detection and molecular graph prediction. This paper considers predicting multiple node labeling functions on graphs simultaneously and revisits this problem from a multitask learning perspective. We develop an algorithm to cluster tasks into groups based on a higher-order task affinity measure.
• Score: 17.70434437597516
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predicting node labels on a given graph is a widely studied problem with many applications, including community detection and molecular graph prediction. This paper considers predicting multiple node labeling functions on graphs simultaneously and revisits this problem from a multitask learning perspective. For a concrete example, consider overlapping community detection: each community membership is a binary node classification task. Due to complex overlapping patterns, we find that negative transfer is prevalent when we apply naive multitask learning to multiple community detection, as task relationships are highly nonlinear across different node labeling. To address the challenge, we develop an algorithm to cluster tasks into groups based on a higher-order task affinity measure. We then fit a multitask model on each task group, resulting in a boosting procedure on top of the baseline model. We estimate the higher-order task affinity measure between two tasks as the prediction loss of one task in the presence of another task and a random subset of other tasks. Then, we use spectral clustering on the affinity score matrix to identify task grouping. We design several speedup techniques to compute the higher-order affinity scores efficiently and show that they can predict negative transfers more accurately than pairwise task affinities. We validate our procedure using various community detection and molecular graph prediction data sets, showing favorable results compared with existing methods. Lastly, we provide a theoretical analysis to show that under a planted block model of tasks on graphs, our affinity scores can provably separate tasks into groups.

Related papers

• Identifying Task Groupings for Multi-Task Learning Using Pointwise V-Usable Information [11.545160697026514]
  We propose a metric of task relatedness based on task difficulty measured by pointwise V-usable information (PVI) We conduct experiments to evaluate the feasibility of this metric for task grouping on 15 NLP datasets in the general, biomedical, and clinical domains.
  arXiv  Detail & Related papers  (2024-10-16T17:49:45Z)
• Scalable Multitask Learning Using Gradient-based Estimation of Task Affinity [16.643892206707854]
  Grad-TAG can estimate task affinities without repeatedly training on data from various task combinations. We show that Grad-TAG achieves excellent performance and runtime tradeoffs compared to existing approaches.
  arXiv  Detail & Related papers  (2024-09-09T21:59:27Z)
• Decoupling Weighing and Selecting for Integrating Multiple Graph Pre-training Tasks [58.65410800008769]
  This paper proposes a novel instance-level framework for integrating multiple graph pre-training tasks, Weigh And Select (WAS) It first adaptively learns an optimal combination of tasks for each instance from a given task pool, based on which a customized instance-level task weighing strategy is learned. Experiments on 16 graph datasets across node-level and graph-level downstream tasks have demonstrated that WAS can achieve comparable performance to other leading counterparts.
  arXiv  Detail & Related papers  (2024-03-03T05:29:49Z)
• ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt [67.8934749027315]
  We propose a unified framework for graph hybrid pre-training which injects the task identification and position identification into GNNs. We also propose a novel pre-training paradigm based on a group of $k$-nearest neighbors.
  arXiv  Detail & Related papers  (2023-10-23T12:11:13Z)
• Task Compass: Scaling Multi-task Pre-training with Task Prefix [122.49242976184617]
  Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks. We propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks. Our model can not only serve as the strong foundation backbone for a wide range of tasks but also be feasible as a probing tool for analyzing task relationships.
  arXiv  Detail & Related papers  (2022-10-12T15:02:04Z)
• Association Graph Learning for Multi-Task Classification with Category Shifts [68.58829338426712]
  We focus on multi-task classification, where related classification tasks share the same label space and are learned simultaneously. We learn an association graph to transfer knowledge among tasks for missing classes. Our method consistently performs better than representative baselines.
  arXiv  Detail & Related papers  (2022-10-10T12:37:41Z)
• FAITH: Few-Shot Graph Classification with Hierarchical Task Graphs [39.576675425158754]
  Few-shot graph classification aims at predicting classes for graphs, given limited labeled graphs for each class. We propose a novel few-shot learning framework FAITH that captures task correlations via constructing a hierarchical task graph. Experiments on four prevalent few-shot graph classification datasets demonstrate the superiority of FAITH over other state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-05-05T04:28:32Z)
• Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search [96.31315520244605]
  Arch-Graph is a transferable NAS method that predicts task-specific optimal architectures. We show Arch-Graph's transferability and high sample efficiency across numerous tasks. It is able to find top 0.16% and 0.29% architectures on average on two search spaces under the budget of only 50 models.
  arXiv  Detail & Related papers  (2022-04-12T16:46:06Z)
• Multi-Relational Graph based Heterogeneous Multi-Task Learning in Community Question Answering [28.91133131424694]
  We develop a multi-relational graph based Multi-Task Learning model called Heterogeneous Multi-Task Graph Isomorphism Network (HMTGIN) In each training forward pass, HMTGIN embeds the input CQA forum graph by an extension of Graph Isomorphism Network and skip connections. In the evaluation, the embeddings are shared among different task-specific output layers to make corresponding predictions.
  arXiv  Detail & Related papers  (2021-09-04T03:19:20Z)
• Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimation [95.72606536493548]
  Multi-person pose estimation is challenging because it localizes body keypoints for multiple persons simultaneously. We propose a novel differentiable Hierarchical Graph Grouping (HGG) method to learn the graph grouping in bottom-up multi-person pose estimation task.
  arXiv  Detail & Related papers  (2020-07-23T08:46:22Z)
• How Much and When Do We Need Higher-order Information in Hypergraphs? A Case Study on Hyperedge Prediction [15.912619060150861]
  We propose a method of incrementally representing group interactions using a notion of n-projected graph whose accumulation contains information on up to n-way interactions. As a downstream task, we consider hyperedge prediction, an extension of link prediction, which is a canonical task for evaluating graph models.
  arXiv  Detail & Related papers  (2020-01-30T05:21:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.