RU-Net: Regularized Unrolling Network for Scene Graph Generation
- URL: http://arxiv.org/abs/2205.01297v1
- Date: Tue, 3 May 2022 04:21:15 GMT
- Title: RU-Net: Regularized Unrolling Network for Scene Graph Generation
- Authors: Xin Lin, Changxing Ding, Jing Zhang, Yibing Zhan, Dacheng Tao
- Abstract summary: Scene graph generation (SGG) aims to detect objects and predict the relationships between each pair of objects.
Existing SGG methods usually suffer from several issues, including 1) ambiguous object representations, and 2) low diversity in relationship predictions.
We propose a regularized unrolling network (RU-Net) to address both problems.
- Score: 92.95032610978511
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Scene graph generation (SGG) aims to detect objects and predict the
relationships between each pair of objects. Existing SGG methods usually suffer
from several issues, including 1) ambiguous object representations, as graph
neural network-based message passing (GMP) modules are typically sensitive to
spurious inter-node correlations, and 2) low diversity in relationship
predictions due to severe class imbalance and a large number of missing
annotations. To address both problems, in this paper, we propose a regularized
unrolling network (RU-Net). We first study the relation between GMP and graph
Laplacian denoising (GLD) from the perspective of the unrolling technique,
determining that GMP can be formulated as a solver for GLD. Based on this
observation, we propose an unrolled message passing module and introduce an
$\ell_p$-based graph regularization to suppress spurious connections between
nodes. Second, we propose a group diversity enhancement module that promotes
the prediction diversity of relationships via rank maximization. Systematic
experiments demonstrate that RU-Net is effective under a variety of settings
and metrics. Furthermore, RU-Net achieves new state-of-the-arts on three
popular databases: VG, VRD, and OI. Code is available at
https://github.com/siml3/RU-Net.
Related papers
- RBA-GCN: Relational Bilevel Aggregation Graph Convolutional Network for
Emotion Recognition [38.87080348908327]
We present the relational bilevel aggregation graph convolutional network (RBA-GCN)
It consists of three modules: the graph generation module (GGM), similarity-based cluster building module (SCBM) and bilevel aggregation module (BiAM)
On both the IEMOCAP and MELD datasets, the weighted average F1 score of RBA-GCN has a 2.17$sim$5.21% improvement over that of the most advanced method.
arXiv Detail & Related papers (2023-08-18T11:29:12Z) - HL-Net: Heterophily Learning Network for Scene Graph Generation [90.2766568914452]
We propose a novel Heterophily Learning Network (HL-Net) to explore the homophily and heterophily between objects/relationships in scene graphs.
HL-Net comprises the following 1) an adaptive reweighting transformer module, which adaptively integrates the information from different layers to exploit both the heterophily and homophily in objects.
We conducted extensive experiments on two public datasets: Visual Genome (VG) and Open Images (OI)
arXiv Detail & Related papers (2022-05-03T06:00:29Z) - Relation Regularized Scene Graph Generation [206.76762860019065]
Scene graph generation (SGG) is built on top of detected objects to predict object pairwise visual relations.
We propose a relation regularized network (R2-Net) which can predict whether there is a relationship between two objects.
Our R2-Net can effectively refine object labels and generate scene graphs.
arXiv Detail & Related papers (2022-02-22T11:36:49Z) - VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using
Vector Quantization [70.8567058758375]
VQ-GNN is a universal framework to scale up any convolution-based GNNs using Vector Quantization (VQ) without compromising the performance.
Our framework avoids the "neighbor explosion" problem of GNNs using quantized representations combined with a low-rank version of the graph convolution matrix.
arXiv Detail & Related papers (2021-10-27T11:48:50Z) - Hierarchical graph neural nets can capture long-range interactions [8.067880298298185]
We study hierarchical message passing models that leverage a multi-resolution representation of a given graph.
This facilitates learning of features that span large receptive fields without loss of local information.
We introduce Hierarchical Graph Net (HGNet), which for any two connected nodes guarantees existence of message-passing paths of at most logarithmic length.
arXiv Detail & Related papers (2021-07-15T16:24:22Z) - Policy-GNN: Aggregation Optimization for Graph Neural Networks [60.50932472042379]
Graph neural networks (GNNs) aim to model the local graph structures and capture the hierarchical patterns by aggregating the information from neighbors.
It is a challenging task to develop an effective aggregation strategy for each node, given complex graphs and sparse features.
We propose Policy-GNN, a meta-policy framework that models the sampling procedure and message passing of GNNs into a combined learning process.
arXiv Detail & Related papers (2020-06-26T17:03:06Z) - GPS-Net: Graph Property Sensing Network for Scene Graph Generation [91.60326359082408]
Scene graph generation (SGG) aims to detect objects in an image along with their pairwise relationships.
GPS-Net fully explores three properties for SGG: edge direction information, the difference in priority between nodes, and the long-tailed distribution of relationships.
GPS-Net achieves state-of-the-art performance on three popular databases: VG, OI, and VRD by significant gains under various settings and metrics.
arXiv Detail & Related papers (2020-03-29T07:22:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.