Frequency-guided Multi-level Reasoning for Scene Graph Generation in Video

• URL: http://arxiv.org/abs/2604.17298v1
• Date: Sun, 19 Apr 2026 07:28:39 GMT
• Title: Frequency-guided Multi-level Reasoning for Scene Graph Generation in Video
• Abstract summary: Video Scene Graph aims to obtain structured representations of objects and their semantic relationships in videos for high-level distributions.<n>This paper proposes Multilevel Reasoning (FReMuRe) model, which enhances the modeling ability of long-tail relationships.<n> Experimental results show that FReMuRe significantly improves the recall rate of long-tail relationships and overall reasoning on the Action Genome dataset.
• Score: 23.512115464610726
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video Scene Graph Generation aims to obtain structured semantic representations of objects and their relationships in videos for high-level understanding. However, existing methods still have limitations in handling long-tail distributions. This paper proposes the Frequency-guided Relational Multi-level Reasoning (FReMuRe) model, which enhances the modeling ability of long-tail relationships from a mechanism perspective. We introduce relation-specific branches to deal gradient conflicts, yielding more balanced and tail-aware learning. And we design a frequency-aware dual-branch predicate embedding network to model high-frequency and low-frequency relationships separately and improve the recall rate of tail classes through gated fusion. Meanwhile, we propose two types of interchangeable relation classification heads: Bayesian Head for uncertainty estimation and new Gaussian Mixture Model Head to enhance intra-class diversity. Experimental results show that FReMuRe significantly improves the recall rate of long-tail relationships and overall reasoning robustness on the Action Genome dataset.

Related papers

• Long-tailed Recognition with Model Rebalancing [34.149401963172934]
  Long-tailed recognition is ubiquitous and challenging in deep learning.<n>We propose a novel framework, Model Rebalancing, which mitigates imbalance by directly rebalancing the model's parameter space.<n> MORE significantly improves generalization, particularly for tail classes, and effectively complements existing imbalance mitigation methods.
  arXiv  Detail & Related papers  (2025-10-09T13:04:38Z)
• SDGF: Fusing Static and Multi-Scale Dynamic Correlations for Multivariate Time Series Forecasting [9.027814258970684]
  Inter-series correlations are crucial for accurate time series forecasting.<n>These relationships often exhibit complex dynamics across different temporal scales.<n>Existing methods are limited in modeling these multi-scale dependencies.
  arXiv  Detail & Related papers  (2025-09-14T11:23:12Z)
• Multi-Level Collaboration in Model Merging [56.31088116526825]
  This paper explores the intrinsic connections between model merging and model ensembling.<n>We find that even when previous restrictions are not met, there is still a way for model merging to attain a near-identical and superior performance similar to that of ensembling.
  arXiv  Detail & Related papers  (2025-03-03T07:45:04Z)
• Robust Training of Federated Models with Extremely Label Deficiency [84.00832527512148]
  Federated semi-supervised learning (FSSL) has emerged as a powerful paradigm for collaboratively training machine learning models using distributed data with label deficiency. We propose a novel twin-model paradigm, called Twin-sight, designed to enhance mutual guidance by providing insights from different perspectives of labeled and unlabeled data. Our comprehensive experiments on four benchmark datasets provide substantial evidence that Twin-sight can significantly outperform state-of-the-art methods across various experimental settings.
  arXiv  Detail & Related papers  (2024-02-22T10:19:34Z)
• Local Consensus Enhanced Siamese Network with Reciprocal Loss for Two-view Correspondence Learning [35.5851523517487]
  Two-view correspondence learning usually establish an end-to-end network to jointly predict correspondence reliability and relative pose. We propose a Local Feature Consensus (LFC) plugin block to augment the features of existing models. We extend existing models to a Siamese network with a reciprocal loss that exploits the supervision of mutual projection.
  arXiv  Detail & Related papers  (2023-08-06T22:20:09Z)
• An Empirical Study: Extensive Deep Temporal Point Process [61.14164208094238]
  We first review recent research emphasis and difficulties in modeling asynchronous event sequences with deep temporal point process.<n>We propose a Granger causality discovery framework for exploiting the relations among multi-types of events.
  arXiv  Detail & Related papers  (2021-10-19T10:15:00Z)
• Learning of Visual Relations: The Devil is in the Tails [59.737494875502215]
  Visual relation learning is a long-tailed problem, due to the nature of joint reasoning about groups of objects. In this paper, we explore an alternative hypothesis, denoted the Devil is in the Tails. Under this hypothesis, better performance is achieved by keeping the model simple but improving its ability to cope with long-tailed distributions.
  arXiv  Detail & Related papers  (2021-08-22T08:59:35Z)
• Distantly-Supervised Long-Tailed Relation Extraction Using Constraint Graphs [16.671606030727975]
  In this paper, we introduce constraint graphs to model the dependencies between relation labels. We also propose a novel constraint graph-based relation extraction framework(CGRE) to handle the two challenges simultaneously. CGRE employs graph convolution networks (GCNs) to propagate information from data-rich relation nodes to data-poor relation nodes.
  arXiv  Detail & Related papers  (2021-05-24T12:02:32Z)
• RH-Net: Improving Neural Relation Extraction via Reinforcement Learning and Hierarchical Relational Searching [2.1828601975620257]
  We propose a novel framework named RH-Net, which utilizes Reinforcement learning and Hierarchical relational searching module to improve relation extraction. We then propose the hierarchical relational searching module to share the semantics from correlative instances between data-rich and data-poor classes.
  arXiv  Detail & Related papers  (2020-10-27T12:50:27Z)
• Improving Long-Tail Relation Extraction with Collaborating Relation-Augmented Attention [63.26288066935098]
  We propose a novel neural network, Collaborating Relation-augmented Attention (CoRA), to handle both the wrong labeling and long-tail relations. In the experiments on the popular benchmark dataset NYT, the proposed CoRA improves the prior state-of-the-art performance by a large margin.
  arXiv  Detail & Related papers  (2020-10-08T05:34:43Z)
• Long-tailed Recognition by Routing Diverse Distribution-Aware Experts [64.71102030006422]
  We propose a new long-tailed classifier called RoutIng Diverse Experts (RIDE) It reduces the model variance with multiple experts, reduces the model bias with a distribution-aware diversity loss, reduces the computational cost with a dynamic expert routing module. RIDE outperforms the state-of-the-art by 5% to 7% on CIFAR100-LT, ImageNet-LT and iNaturalist 2018 benchmarks.
  arXiv  Detail & Related papers  (2020-10-05T06:53:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.