Exploring the Hierarchy in Relation Labels for Scene Graph Generation
- URL: http://arxiv.org/abs/2009.05834v1
- Date: Sat, 12 Sep 2020 17:36:53 GMT
- Title: Exploring the Hierarchy in Relation Labels for Scene Graph Generation
- Authors: Yi Zhou, Shuyang Sun, Chao Zhang, Yikang Li, Wanli Ouyang
- Abstract summary: The proposed method can improve several state-of-the-art baselines by a large margin (up to $33%$ relative gain) in terms of Recall@50.
Experiments show that the proposed simple yet effective method can improve several state-of-the-art baselines by a large margin.
- Score: 75.88758055269948
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: By assigning each relationship a single label, current approaches formulate
the relationship detection as a classification problem. Under this formulation,
predicate categories are treated as completely different classes. However,
different from the object labels where different classes have explicit
boundaries, predicates usually have overlaps in their semantic meanings. For
example, sit\_on and stand\_on have common meanings in vertical relationships
but different details of how these two objects are vertically placed. In order
to leverage the inherent structures of the predicate categories, we propose to
first build the language hierarchy and then utilize the Hierarchy Guided
Feature Learning (HGFL) strategy to learn better region features of both the
coarse-grained level and the fine-grained level. Besides, we also propose the
Hierarchy Guided Module (HGM) to utilize the coarse-grained level to guide the
learning of fine-grained level features. Experiments show that the proposed
simple yet effective method can improve several state-of-the-art baselines by a
large margin (up to $33\%$ relative gain) in terms of Recall@50 on the task of
Scene Graph Generation in different datasets.
Related papers
- HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text
Classification [19.12354692458442]
Hierarchical text classification (HTC) is a complex subtask under multi-label text classification.
We propose HiGen, a text-generation-based framework utilizing language models to encode dynamic text representations.
arXiv Detail & Related papers (2024-01-24T04:44:42Z) - SEAL: Simultaneous Label Hierarchy Exploration And Learning [9.701914280306118]
We propose a new framework that explores the label hierarchy by augmenting the observed labels with latent labels that follow a prior hierarchical structure.
Our approach uses a 1-Wasserstein metric over the tree metric space as an objective function, which enables us to simultaneously learn a data-driven label hierarchy and perform (semi-supervised) learning.
arXiv Detail & Related papers (2023-04-26T08:31:59Z) - Exploiting Global and Local Hierarchies for Hierarchical Text
Classification [34.624922210257125]
Existing methods encode label hierarchy in a global view, where label hierarchy is treated as the static hierarchical structure containing all labels.
We propose Hierarchy-guided BERT with Global and Local hierarchies (HBGL) to model both global and local hierarchies.
Compared with the state-of-the-art method HGCLR, our method achieves significant improvement on three benchmark datasets.
arXiv Detail & Related papers (2022-05-05T12:48:41Z) - Use All The Labels: A Hierarchical Multi-Label Contrastive Learning
Framework [75.79736930414715]
We present a hierarchical multi-label representation learning framework that can leverage all available labels and preserve the hierarchical relationship between classes.
We introduce novel hierarchy preserving losses, which jointly apply a hierarchical penalty to the contrastive loss, and enforce the hierarchy constraint.
arXiv Detail & Related papers (2022-04-27T21:41:44Z) - Label Hierarchy Transition: Delving into Class Hierarchies to Enhance
Deep Classifiers [40.993137740456014]
We propose a unified probabilistic framework based on deep learning to address the challenges of hierarchical classification.
The proposed framework can be readily adapted to any existing deep network with only minor modifications.
We extend our proposed LHT framework to the skin lesion diagnosis task and validate its great potential in computer-aided diagnosis.
arXiv Detail & Related papers (2021-12-04T14:58:36Z) - Hierarchical Heterogeneous Graph Representation Learning for Short Text
Classification [60.233529926965836]
We propose a new method called SHINE, which is based on graph neural network (GNN) for short text classification.
First, we model the short text dataset as a hierarchical heterogeneous graph consisting of word-level component graphs.
Then, we dynamically learn a short document graph that facilitates effective label propagation among similar short texts.
arXiv Detail & Related papers (2021-10-30T05:33:05Z) - MATCH: Metadata-Aware Text Classification in A Large Hierarchy [60.59183151617578]
MATCH is an end-to-end framework that leverages both metadata and hierarchy information.
We propose different ways to regularize the parameters and output probability of each child label by its parents.
Experiments on two massive text datasets with large-scale label hierarchies demonstrate the effectiveness of MATCH.
arXiv Detail & Related papers (2021-02-15T05:23:08Z) - Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer [140.72439827136085]
We propose a graph reasoning and transfer learning framework named "Graphonomy"
It incorporates human knowledge and label taxonomy into the intermediate graph representation learning beyond local convolutions.
It learns the global and structured semantic coherency in multiple domains via semantic-aware graph reasoning and transfer.
arXiv Detail & Related papers (2021-01-26T08:19:03Z) - Joint Learning of Hyperbolic Label Embeddings for Hierarchical
Multi-label Classification [9.996804039553858]
We consider the problem of multi-label classification where the labels lie in a hierarchy.
We propose a novel formulation for the joint learning and empirically evaluate its efficacy.
arXiv Detail & Related papers (2021-01-13T10:58:54Z) - Hierarchical Image Classification using Entailment Cone Embeddings [68.82490011036263]
We first inject label-hierarchy knowledge into an arbitrary CNN-based classifier.
We empirically show that availability of such external semantic information in conjunction with the visual semantics from images boosts overall performance.
arXiv Detail & Related papers (2020-04-02T10:22:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.