Graph-Based Similarity of Neural Network Representations

• URL: http://arxiv.org/abs/2111.11165v1
• Date: Mon, 22 Nov 2021 13:04:19 GMT
• Title: Graph-Based Similarity of Neural Network Representations
• Authors: Zuohui Chen, Yao Lu, Wen Yang, Qi Xuan, Xiaoniu Yang
• Abstract summary: We propose Graph-Based Similarity (GBS) to measure the similarity of layer features. By treating each input sample as a node and the corresponding layer output similarity as edges, we construct the graph of representations for each layer. GBS shows state-of-the-art performance in reflecting the similarity and provides insights on explaining the adversarial sample behavior on the hidden layer space.
• Score: 8.424772972066696
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding the black-box representations in Deep Neural Networks (DNN) is an essential problem in deep learning. In this work, we propose Graph-Based Similarity (GBS) to measure the similarity of layer features. Contrary to previous works that compute the similarity directly on the feature maps, GBS measures the correlation based on the graph constructed with hidden layer outputs. By treating each input sample as a node and the corresponding layer output similarity as edges, we construct the graph of DNN representations for each layer. The similarity between graphs of layers identifies the correspondences between representations of models trained in different datasets and initializations. We demonstrate and prove the invariance property of GBS, including invariance to orthogonal transformation and invariance to isotropic scaling, and compare GBS with CKA. GBS shows state-of-the-art performance in reflecting the similarity and provides insights on explaining the adversarial sample behavior on the hidden layer space.

Related papers

• Exploring Consistency in Graph Representations:from Graph Kernels to Graph Neural Networks [4.235378870514331]
  Graph Networks (GNNs) have emerged as a dominant approach in graph representation learning. We bridge the gap between neural network methods and kernel approaches by enabling GNNs to consistently capture structures in their learned representations. Inspired by these findings, we conjecture that the consistency in the similarities of graph representations across GNN layers is crucial in capturing relational structures and enhancing graph classification performance.
  arXiv  Detail & Related papers  (2024-10-31T09:07:08Z)
• An Ad-hoc graph node vector embedding algorithm for general knowledge graphs using Kinetica-Graph [0.0]
  This paper discusses how to generate general graph node embeddings from knowledge graph representations. The embedded space is composed of a number of sub-features to mimic both local affinity and remote structural relevance.
  arXiv  Detail & Related papers  (2024-07-22T14:43:10Z)
• Pairwise Alignment Improves Graph Domain Adaptation [16.626928606474173]
  This work delves into Graph Domain Adaptation (GDA) to address the unique complexities of distribution shifts over graph data. We propose a novel, theoretically principled method, Pairwise Alignment (Pair-Align) to counter graph structure shift. Our method demonstrates superior performance in real-world applications, including node classification with region shift in social networks.
  arXiv  Detail & Related papers  (2024-03-02T04:31:28Z)
• SimCalib: Graph Neural Network Calibration based on Similarity between Nodes [60.92081159963772]
  Graph neural networks (GNNs) have exhibited impressive performance in modeling graph data as exemplified in various applications. We shed light on the relationship between GNN calibration and nodewise similarity via theoretical analysis. A novel calibration framework, named SimCalib, is accordingly proposed to consider similarity between nodes at global and local levels.
  arXiv  Detail & Related papers  (2023-12-19T04:58:37Z)
• CGMN: A Contrastive Graph Matching Network for Self-Supervised Graph Similarity Learning [65.1042892570989]
  We propose a contrastive graph matching network (CGMN) for self-supervised graph similarity learning. We employ two strategies, namely cross-view interaction and cross-graph interaction, for effective node representation learning. We transform node representations into graph-level representations via pooling operations for graph similarity computation.
  arXiv  Detail & Related papers  (2022-05-30T13:20:26Z)
• Learning Hierarchical Graph Representation for Image Manipulation Detection [50.04902159383709]
  The objective of image manipulation detection is to identify and locate the manipulated regions in the images. Recent approaches mostly adopt the sophisticated Convolutional Neural Networks (CNNs) to capture the tampering artifacts left in the images. We propose a hierarchical Graph Convolutional Network (HGCN-Net), which consists of two parallel branches.
  arXiv  Detail & Related papers  (2022-01-15T01:54:25Z)
• Graph Neural Networks with Feature and Structure Aware Random Walk [7.143879014059894]
  We show that in typical heterphilous graphs, the edges may be directed, and whether to treat the edges as is or simply make them undirected greatly affects the performance of the GNN models. We develop a model that adaptively learns the directionality of the graph, and exploits the underlying long-distance correlations between nodes.
  arXiv  Detail & Related papers  (2021-11-19T08:54:21Z)
• Explicit Pairwise Factorized Graph Neural Network for Semi-Supervised Node Classification [59.06717774425588]
  We propose the Explicit Pairwise Factorized Graph Neural Network (EPFGNN), which models the whole graph as a partially observed Markov Random Field. It contains explicit pairwise factors to model output-output relations and uses a GNN backbone to model input-output relations. We conduct experiments on various datasets, which shows that our model can effectively improve the performance for semi-supervised node classification on graphs.
  arXiv  Detail & Related papers  (2021-07-27T19:47:53Z)
• Spectral-Spatial Global Graph Reasoning for Hyperspectral Image Classification [50.899576891296235]
  Convolutional neural networks have been widely applied to hyperspectral image classification. Recent methods attempt to address this issue by performing graph convolutions on spatial topologies.
  arXiv  Detail & Related papers  (2021-06-26T06:24:51Z)
• Interpretable Visualizations with Differentiating Embedding Networks [0.0]
  We present a visualization algorithm based on a novel unsupervised Siamese neural network training regime and loss function, called Differentiating Embedding Networks (DEN) The Siamese neural network finds differentiating or similar features between specific pairs of samples in a dataset, and uses these features to embed the dataset in a lower dimensional space where it can be visualized. To interpret DEN, we create an end-to-end parametric clustering algorithm on top of the visualization, and then leverage SHAP scores to determine which features in the sample space are important for understanding the structures shown in the visualization based on the clusters found.
  arXiv  Detail & Related papers  (2020-06-11T17:30:44Z)
• Graphon Pooling in Graph Neural Networks [169.09536309161314]
  Graph neural networks (GNNs) have been used effectively in different applications involving the processing of signals on irregular structures modeled by graphs. We propose a new strategy for pooling and sampling on GNNs using graphons which preserves the spectral properties of the graph.
  arXiv  Detail & Related papers  (2020-03-03T21:04:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.