Dynamic Graph: Learning Instance-aware Connectivity for Neural Networks
- URL: http://arxiv.org/abs/2010.01097v1
- Date: Fri, 2 Oct 2020 16:50:26 GMT
- Title: Dynamic Graph: Learning Instance-aware Connectivity for Neural Networks
- Authors: Kun Yuan, Quanquan Li, Dapeng Chen, Aojun Zhou and Junjie Yan
- Abstract summary: Dynamic Graph Network (DG-Net) is a complete directed acyclic graph, where the nodes represent convolutional blocks and the edges represent connection paths.
Instead of using the same path of the network, DG-Net aggregates features dynamically in each node, which allows the network to have more representation ability.
- Score: 78.65792427542672
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: One practice of employing deep neural networks is to apply the same
architecture to all the input instances. However, a fixed architecture may not
be representative enough for data with high diversity. To promote the model
capacity, existing approaches usually employ larger convolutional kernels or
deeper network structure, which may increase the computational cost. In this
paper, we address this issue by raising the Dynamic Graph Network (DG-Net). The
network learns the instance-aware connectivity, which creates different forward
paths for different instances. Specifically, the network is initialized as a
complete directed acyclic graph, where the nodes represent convolutional blocks
and the edges represent the connection paths. We generate edge weights by a
learnable module \textit{router} and select the edges whose weights are larger
than a threshold, to adjust the connectivity of the neural network structure.
Instead of using the same path of the network, DG-Net aggregates features
dynamically in each node, which allows the network to have more representation
ability. To facilitate the training, we represent the network connectivity of
each sample in an adjacency matrix. The matrix is updated to aggregate features
in the forward pass, cached in the memory, and used for gradient computing in
the backward pass. We verify the effectiveness of our method with several
static architectures, including MobileNetV2, ResNet, ResNeXt, and RegNet.
Extensive experiments are performed on ImageNet classification and COCO object
detection, which shows the effectiveness and generalization ability of our
approach.
Related papers
- Online Learning Of Expanding Graphs [14.952056744888916]
This paper addresses the problem of online network inference for expanding graphs from a stream of signals.
We introduce a strategy that enables different types of updates for nodes that just joined the network and for previously existing nodes.
arXiv Detail & Related papers (2024-09-13T09:20:42Z) - Graph Metanetworks for Processing Diverse Neural Architectures [33.686728709734105]
Graph Metanetworks (GMNs) generalizes to neural architectures where competing methods struggle.
We prove that GMNs are expressive and equivariant to parameter permutation symmetries that leave the input neural network functions.
arXiv Detail & Related papers (2023-12-07T18:21:52Z) - Infinite Width Graph Neural Networks for Node Regression/ Classification [0.0]
This work analyzes Graph Neural Networks, a generalization of Fully-Connected Deep Neural Nets on Graph structured data, when their width, that is the number of nodes in each fullyconnected layer is increasing to infinity.
arXiv Detail & Related papers (2023-10-12T10:01:39Z) - Network Alignment with Transferable Graph Autoencoders [79.89704126746204]
We propose a novel graph autoencoder architecture designed to extract powerful and robust node embeddings.
We prove that the generated embeddings are associated with the eigenvalues and eigenvectors of the graphs.
Our proposed framework also leverages transfer learning and data augmentation to achieve efficient network alignment at a very large scale without retraining.
arXiv Detail & Related papers (2023-10-05T02:58:29Z) - EGRC-Net: Embedding-induced Graph Refinement Clustering Network [66.44293190793294]
We propose a novel graph clustering network called Embedding-Induced Graph Refinement Clustering Network (EGRC-Net)
EGRC-Net effectively utilizes the learned embedding to adaptively refine the initial graph and enhance the clustering performance.
Our proposed methods consistently outperform several state-of-the-art approaches.
arXiv Detail & Related papers (2022-11-19T09:08:43Z) - Dynamic Graph Message Passing Networks for Visual Recognition [112.49513303433606]
Modelling long-range dependencies is critical for scene understanding tasks in computer vision.
A fully-connected graph is beneficial for such modelling, but its computational overhead is prohibitive.
We propose a dynamic graph message passing network, that significantly reduces the computational complexity.
arXiv Detail & Related papers (2022-09-20T14:41:37Z) - Temporal Graph Network Embedding with Causal Anonymous Walks
Representations [54.05212871508062]
We propose a novel approach for dynamic network representation learning based on Temporal Graph Network.
For evaluation, we provide a benchmark pipeline for the evaluation of temporal network embeddings.
We show the applicability and superior performance of our model in the real-world downstream graph machine learning task provided by one of the top European banks.
arXiv Detail & Related papers (2021-08-19T15:39:52Z) - Adjoined Networks: A Training Paradigm with Applications to Network
Compression [3.995047443480282]
We introduce Adjoined Networks, or AN, a learning paradigm that trains both the original base network and the smaller compressed network together.
Using ResNet-50 as the base network, AN achieves 71.8% top-1 accuracy with only 1.8M parameters and 1.6 GFLOPs on the ImageNet data-set.
We propose Differentiable Adjoined Networks (DAN), a training paradigm that augments AN by using neural architecture search to jointly learn both the width and the weights for each layer of the smaller network.
arXiv Detail & Related papers (2020-06-10T02:48:16Z) - EdgeNets:Edge Varying Graph Neural Networks [179.99395949679547]
This paper puts forth a general framework that unifies state-of-the-art graph neural networks (GNNs) through the concept of EdgeNet.
An EdgeNet is a GNN architecture that allows different nodes to use different parameters to weigh the information of different neighbors.
This is a general linear and local operation that a node can perform and encompasses under one formulation all existing graph convolutional neural networks (GCNNs) as well as graph attention networks (GATs)
arXiv Detail & Related papers (2020-01-21T15:51:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.