RawlsGCN: Towards Rawlsian Difference Principle on Graph Convolutional Network

• URL: http://arxiv.org/abs/2202.13547v1
• Date: Mon, 28 Feb 2022 05:07:57 GMT
• Title: RawlsGCN: Towards Rawlsian Difference Principle on Graph Convolutional Network
• Authors: Jian Kang, Yan Zhu, Yinglong Xia, Jiebo Luo, Hanghang Tong
• Abstract summary: Graph Convolutional Network (GCN) plays pivotal roles in many real-world applications. GCN often exhibits performance disparity with respect to node degrees, resulting in worse predictive accuracy for low-degree nodes. We formulate the problem of mitigating the degree-related performance disparity in GCN from the perspective of the Rawlsian difference principle.
• Score: 102.27090022283208
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph Convolutional Network (GCN) plays pivotal roles in many real-world applications. Despite the successes of GCN deployment, GCN often exhibits performance disparity with respect to node degrees, resulting in worse predictive accuracy for low-degree nodes. We formulate the problem of mitigating the degree-related performance disparity in GCN from the perspective of the Rawlsian difference principle, which is originated from the theory of distributive justice. Mathematically, we aim to balance the utility between low-degree nodes and high-degree nodes while minimizing the task-specific loss. Specifically, we reveal the root cause of this degree-related unfairness by analyzing the gradients of weight matrices in GCN. Guided by the gradients of weight matrices, we further propose a pre-processing method RawlsGCN-Graph and an in-processing method RawlsGCN-Grad that achieves fair predictive accuracy in low-degree nodes without modification on the GCN architecture or introduction of additional parameters. Extensive experiments on real-world graphs demonstrate the effectiveness of our proposed RawlsGCN methods in significantly reducing degree-related bias while retaining comparable overall performance.

Related papers

• OrthoReg: Improving Graph-regularized MLPs via Orthogonality Regularization [66.30021126251725]
  Graph Neural Networks (GNNs) are currently dominating in modeling graphstructure data. Graph-regularized networks (GR-MLPs) implicitly inject the graph structure information into model weights, while their performance can hardly match that of GNNs in most tasks. We show that GR-MLPs suffer from dimensional collapse, a phenomenon in which the largest a few eigenvalues dominate the embedding space. We propose OrthoReg, a novel GR-MLP model to mitigate the dimensional collapse issue.
  arXiv  Detail & Related papers  (2023-01-31T21:20:48Z)
• Uncovering the Structural Fairness in Graph Contrastive Learning [87.65091052291544]
  Graph contrastive learning (GCL) has emerged as a promising self-supervised approach for learning node representations. We show that representations obtained by GCL methods are already fairer to degree bias than those learned by GCN. We devise a novel graph augmentation method, called GRAph contrastive learning for DEgree bias (GRADE), which applies different strategies to low- and high-degree nodes.
  arXiv  Detail & Related papers  (2022-10-06T15:58:25Z)
• ResNorm: Tackling Long-tailed Degree Distribution Issue in Graph Neural Networks via Normalization [80.90206641975375]
  This paper focuses on improving the performance of GNNs via normalization. By studying the long-tailed distribution of node degrees in the graph, we propose a novel normalization method for GNNs. The $scale$ operation of ResNorm reshapes the node-wise standard deviation (NStd) distribution so as to improve the accuracy of tail nodes.
  arXiv  Detail & Related papers  (2022-06-16T13:49:09Z)
• Graph Partner Neural Networks for Semi-Supervised Learning on Graphs [16.489177915147785]
  Graph Convolutional Networks (GCNs) are powerful for processing graphstructured data and have achieved state-of-the-art performance in several tasks such as node classification, link prediction, and graph classification. It is inevitable for deep GCNs to suffer from an over-smoothing issue that the representations of nodes will tend to be indistinguishable after repeated graph convolution operations. We propose the Graph Partner Neural Network (GPNN) which incorporates a de- parameterized GCN and a parameter-sharing scheme.
  arXiv  Detail & Related papers  (2021-10-18T10:56:56Z)
• Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective [10.178145000390671]
  Graph convolutional networks (GCNs) have achieved promising performance on various graph-based tasks. However they suffer from over-smoothing when stacking more layers. We present a quantitative study on this observation and develop novel insights towards the deeper GCN.
  arXiv  Detail & Related papers  (2020-09-24T03:36:43Z)
• Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks [62.8504260693664]
  Graph Convolutional Networks (GCNs) show promising results for semisupervised learning tasks on graphs. In this paper, we analyze GCNs in regard to the node degree distribution. We develop a novel Self-Supervised DegreeSpecific GCN (SL-DSGC) that mitigates the degree biases of GCNs.
  arXiv  Detail & Related papers  (2020-06-28T16:26:47Z)
• Understanding and Resolving Performance Degradation in Graph Convolutional Networks [105.14867349802898]
  Graph Convolutional Network (GCN) stacks several layers and in each layer performs a PROPagation operation (PROP) and a TRANsformation operation (TRAN) for learning node representations over graph-structured data. GCNs tend to suffer performance drop when the model gets deep. We study performance degradation of GCNs by experimentally examining how stacking only TRANs or PROPs works.
  arXiv  Detail & Related papers  (2020-06-12T12:12:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.