Adaptive Fine-Grained Predicates Learning for Scene Graph Generation
- URL: http://arxiv.org/abs/2207.04602v1
- Date: Mon, 11 Jul 2022 03:37:57 GMT
- Title: Adaptive Fine-Grained Predicates Learning for Scene Graph Generation
- Authors: Xinyu Lyu, Lianli Gao, Pengpeng Zeng, Heng Tao Shen, Jingkuan Song
- Abstract summary: General Scene Graph Generation (SGG) models tend to predict head predicates and re-balancing strategies prefer tail categories.
We propose an Adaptive Fine-Grained Predicates Learning (FGPL-A) which aims at differentiating hard-to-distinguish predicates for SGG.
Our proposed model-agnostic strategy significantly boosts performance of benchmark models on VG-SGG and GQA-SGG datasets by up to 175% and 76% on Mean Recall@100, achieving new state-of-the-art performance.
- Score: 122.4588401267544
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The performance of current Scene Graph Generation (SGG) models is severely
hampered by hard-to-distinguish predicates, e.g., woman-on/standing on/walking
on-beach. As general SGG models tend to predict head predicates and
re-balancing strategies prefer tail categories, none of them can appropriately
handle hard-to-distinguish predicates. To tackle this issue, inspired by
fine-grained image classification, which focuses on differentiating
hard-to-distinguish objects, we propose an Adaptive Fine-Grained Predicates
Learning (FGPL-A) which aims at differentiating hard-to-distinguish predicates
for SGG. First, we introduce an Adaptive Predicate Lattice (PL-A) to figure out
hard-to-distinguish predicates, which adaptively explores predicate
correlations in keeping with model's dynamic learning pace. Practically, PL-A
is initialized from SGG dataset, and gets refined by exploring model's
predictions of current mini-batch. Utilizing PL-A, we propose an Adaptive
Category Discriminating Loss (CDL-A) and an Adaptive Entity Discriminating Loss
(EDL-A), which progressively regularize model's discriminating process with
fine-grained supervision concerning model's dynamic learning status, ensuring
balanced and efficient learning process. Extensive experimental results show
that our proposed model-agnostic strategy significantly boosts performance of
benchmark models on VG-SGG and GQA-SGG datasets by up to 175% and 76% on Mean
Recall@100, achieving new state-of-the-art performance. Moreover, experiments
on Sentence-to-Graph Retrieval and Image Captioning tasks further demonstrate
practicability of our method.
Related papers
- Ensemble Predicate Decoding for Unbiased Scene Graph Generation [40.01591739856469]
Scene Graph Generation (SGG) aims to generate a comprehensive graphical representation that captures semantic information of a given scenario.
The model's performance in predicting more fine-grained predicates is hindered by a significant predicate bias.
This paper proposes Ensemble Predicate Decoding (EPD), which employs multiple decoders to attain unbiased scene graph generation.
arXiv Detail & Related papers (2024-08-26T11:24:13Z) - Adaptive Meta-learner via Gradient Similarity for Few-shot Text
Classification [11.035878821365149]
We propose a novel Adaptive Meta-learner via Gradient Similarity (AMGS) to improve the model generalization ability to a new task.
Experimental results on several benchmarks demonstrate that the proposed AMGS consistently improves few-shot text classification performance.
arXiv Detail & Related papers (2022-09-10T16:14:53Z) - Fine-grained Retrieval Prompt Tuning [149.9071858259279]
Fine-grained Retrieval Prompt Tuning steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompt and feature adaptation.
Our FRPT with fewer learnable parameters achieves the state-of-the-art performance on three widely-used fine-grained datasets.
arXiv Detail & Related papers (2022-07-29T04:10:04Z) - Fine-Grained Predicates Learning for Scene Graph Generation [155.48614435437355]
Fine-Grained Predicates Learning aims at differentiating among hard-to-distinguish predicates for Scene Graph Generation task.
We introduce a Predicate Lattice that helps SGG models to figure out fine-grained predicate pairs.
We then propose a Category Discriminating Loss and an Entity Discriminating Loss, which both contribute to distinguishing fine-grained predicates.
arXiv Detail & Related papers (2022-04-06T06:20:09Z) - Hierarchical Memory Learning for Fine-Grained Scene Graph Generation [49.39355372599507]
This paper proposes a novel Hierarchical Memory Learning (HML) framework to learn the model from simple to complex.
After the autonomous partition of coarse and fine predicates, the model is first trained on the coarse predicates and then learns the fine predicates.
arXiv Detail & Related papers (2022-03-14T08:01:14Z) - PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph
Generation [58.98802062945709]
We propose a novel Predicate-Correlation Perception Learning scheme to adaptively seek out appropriate loss weights.
Our PCPL framework is further equipped with a graph encoder module to better extract context features.
arXiv Detail & Related papers (2020-09-02T08:30:09Z) - Few-shot Classification via Adaptive Attention [93.06105498633492]
We propose a novel few-shot learning method via optimizing and fast adapting the query sample representation based on very few reference samples.
As demonstrated experimentally, the proposed model achieves state-of-the-art classification results on various benchmark few-shot classification and fine-grained recognition datasets.
arXiv Detail & Related papers (2020-08-06T05:52:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.