GraPLUS: Graph-based Placement Using Semantics for Image Composition

• URL: http://arxiv.org/abs/2503.15761v1
• Date: Thu, 20 Mar 2025 00:43:29 GMT
• Title: GraPLUS: Graph-based Placement Using Semantics for Image Composition
• Abstract summary: We present GraPLUS (Graph-based Placement Using Semantics), a novel framework for plausible object placement in images.<n>Our approach uniquely combines graph-structured scene representation with semantic understanding to determine contextually appropriate object positions.<n>GraPLUS placement accuracy of 92.1% and an FID score of 28.83 on the OPA dataset, outperforming state-of-the-art methods by 8.1% while maintaining competitive visual quality.
• Score: 3.0450307343472405
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present GraPLUS (Graph-based Placement Using Semantics), a novel framework for plausible object placement in images that leverages scene graphs and large language models. Our approach uniquely combines graph-structured scene representation with semantic understanding to determine contextually appropriate object positions. The framework employs GPT-2 to transform categorical node and edge labels into rich semantic embeddings that capture both definitional characteristics and typical spatial contexts, enabling nuanced understanding of object relationships and placement patterns. GraPLUS achieves placement accuracy of 92.1% and an FID score of 28.83 on the OPA dataset, outperforming state-of-the-art methods by 8.1% while maintaining competitive visual quality. In human evaluation studies involving 964 samples assessed by 19 participants, our method was preferred in 52.1% of cases, significantly outperforming previous approaches. The framework's key innovations include: (i) leveraging pre-trained scene graph models that transfer knowledge from other domains, (ii) edge-aware graph neural networks that process scene semantics through structured relationships, (iii) a cross-modal attention mechanism that aligns categorical embeddings with enhanced scene features, and (iv) a multiobjective training strategy incorporating semantic consistency constraints.

Related papers

• Text-Guided Visual Dependency Graph Learning with Cross-Modal Attention Priors [11.338098263634537]
  We propose CM-GLasso, a framework that bridges vision-language representation learning and sparse Gaussian Models.<n> CM-GLasso introduces three key components: (i) a text visualization strategy that renders class-attribute descriptions as images and processes them through the same SigLIP-2 vision encoder as natural images, (ii) a cross-attention distillation mechanism that condenses high-dimensional patches into a small set of semantic graph nodes, and (iii) a joint ADMM formulation that estimates shared and class-specific precision components within a single convex objective.
  arXiv  Detail & Related papers  (2026-08-19T09:57:34Z)
• With Great Context Comes Great Prediction Power: Classifying Objects via Geo-Semantic Scene Graphs [5.492064811668243]
  This paper argues for the critical role of context and introduces a novel framework for contextual object classification.<n>We first construct a Geo-Semantic Contextual Graph (GSCG) from a single monocular image.<n>This explicit graph structure makes the model's reasoning process inherently interpretable.
  arXiv  Detail & Related papers  (2025-12-28T17:53:55Z)
• Instance-Guided Class Activation Mapping for Weakly Supervised Semantic Segmentation [5.539128209356213]
  We propose IG-CAM, a novel approach to generate high-quality, boundary-aware localization maps.<n>Our approach demonstrates superior localization accuracy, with complete object coverage and precise boundary delineation.<n>The results establish IG-CAM as a new benchmark for weakly supervised semantic segmentation.
  arXiv  Detail & Related papers  (2025-09-15T22:41:44Z)
• Towards Explainable Job Title Matching: Leveraging Semantic Textual Relatedness and Knowledge Graphs [0.19116784879310025]
  This study investigates semantic textual relatedness (STR) in the context of job title matching.<n>We introduce a self-supervised hybrid architecture that combines dense sentence embeddings with domain-specific Knowledge Graphs.<n>We show that fine-tuned SBERT models augmented with KGs produce consistent improvements in the high-STR region.
  arXiv  Detail & Related papers  (2025-09-11T15:02:54Z)
• Language Guided Domain Generalized Medical Image Segmentation [68.93124785575739]
  Single source domain generalization holds promise for more reliable and consistent image segmentation across real-world clinical settings. We propose an approach that explicitly leverages textual information by incorporating a contrastive learning mechanism guided by the text encoder features. Our approach achieves favorable performance against existing methods in literature.
  arXiv  Detail & Related papers  (2024-04-01T17:48:15Z)
• Structure Your Data: Towards Semantic Graph Counterfactuals [1.8817715864806608]
  Counterfactual explanations (CEs) based on concepts are explanations that consider alternative scenarios to understand which high-level semantic features contributed to model predictions. In this work, we propose CEs based on the semantic graphs accompanying input data to achieve more descriptive, accurate, and human-aligned explanations.
  arXiv  Detail & Related papers  (2024-03-11T08:40:37Z)
• A Graph-Based Approach for Category-Agnostic Pose Estimation [12.308036453869033]
  Category-agnostic pose estimation (CAPE) was introduced to enable keypoint localization for arbitrary object categories. We present a significant departure from conventional CAPE techniques, which treat keypoints as isolated entities, by treating the input pose data as a graph. Our solution boosts performance by 0.98% under a 1-shot setting, achieving a new state-of-the-art for CAPE.
  arXiv  Detail & Related papers  (2023-11-29T18:44:12Z)
• Less is More: Toward Zero-Shot Local Scene Graph Generation via Foundation Models [16.08214739525615]
  We present a new task called Local Scene Graph Generation. It aims to abstract pertinent structural information with partial objects and their relationships in an image. We introduce zEro-shot Local scEne GrAph geNeraTion (ELEGANT), a framework harnessing foundation models renowned for their powerful perception and commonsense reasoning.
  arXiv  Detail & Related papers  (2023-10-02T17:19:04Z)
• Learning High-level Semantic-Relational Concepts for SLAM [10.528810470934781]
  We propose an algorithm for learning high-level semantic-relational concepts that can be inferred from the low-level factor graph. We validate our method in both simulated and real datasets demonstrating improved performance over two baseline approaches.
  arXiv  Detail & Related papers  (2023-09-30T14:54:31Z)
• Exploring Open-Vocabulary Semantic Segmentation without Human Labels [76.15862573035565]
  We present ZeroSeg, a novel method that leverages the existing pretrained vision-language model (VL) to train semantic segmentation models. ZeroSeg overcomes this by distilling the visual concepts learned by VL models into a set of segment tokens, each summarizing a localized region of the target image. Our approach achieves state-of-the-art performance when compared to other zero-shot segmentation methods under the same training data.
  arXiv  Detail & Related papers  (2023-06-01T08:47:06Z)
• GPGait: Generalized Pose-based Gait Recognition [11.316545213493223]
  Recent works on pose-based gait recognition have demonstrated the potential of using such simple information to achieve results comparable to silhouette-based methods. To improve the generalization ability of pose-based methods across datasets, we propose a textbfGeneralized textbfPose-based textbfGait recognition framework.
  arXiv  Detail & Related papers  (2023-03-09T13:17:13Z)
• 1st Place Solution for PSG competition with ECCV'22 SenseHuman Workshop [1.5362025549031049]
  Panoptic Scene Graph (PSG) generation aims to generate scene graph representations based on panoptic segmentation instead of rigid bounding boxes. We propose GRNet, a Global Relation Network in two-stage paradigm, where the pre-extracted local object features and their corresponding masks are fed into a transformer with class embeddings. We conduct comprehensive experiments on OpenPSG dataset and achieve the state-of-art performance on the leadboard.
  arXiv  Detail & Related papers  (2023-02-06T09:47:46Z)
• Target-oriented Sentiment Classification with Sequential Cross-modal Semantic Graph [27.77392307623526]
  Multi-modal aspect-based sentiment classification (MABSC) is task of classifying the sentiment of a target entity mentioned in a sentence and an image. Previous methods failed to account for the fine-grained semantic association between the image and the text. We propose a new approach called SeqCSG, which enhances the encoder-decoder sentiment classification framework using sequential cross-modal semantic graphs.
  arXiv  Detail & Related papers  (2022-08-19T16:04:29Z)
• Attention-Guided Supervised Contrastive Learning for Semantic Segmentation [16.729068267453897]
  In a per-pixel prediction task, more than one label can exist in a single image for segmentation. We propose an attention-guided supervised contrastive learning approach to highlight a single semantic object every time as the target.
  arXiv  Detail & Related papers  (2021-06-03T05:01:11Z)
• GINet: Graph Interaction Network for Scene Parsing [58.394591509215005]
  We propose a Graph Interaction unit (GI unit) and a Semantic Context Loss (SC-loss) to promote context reasoning over image regions. The proposed GINet outperforms the state-of-the-art approaches on the popular benchmarks, including Pascal-Context and COCO Stuff.
  arXiv  Detail & Related papers  (2020-09-14T02:52:45Z)
• Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning [85.6386289476598]
  We develop a novel adversarial graph representation adaptation (AGRA) framework for cross-domain holistic-local feature co-adaptation. We conduct extensive and fair evaluations on several popular benchmarks and show that the proposed AGRA framework outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-03T15:00:31Z)
• Adversarial Bipartite Graph Learning for Video Domain Adaptation [50.68420708387015]
  Domain adaptation techniques, which focus on adapting models between distributionally different domains, are rarely explored in the video recognition area. Recent works on visual domain adaptation which leverage adversarial learning to unify the source and target video representations are not highly effective on the videos. This paper proposes an Adversarial Bipartite Graph (ABG) learning framework which directly models the source-target interactions.
  arXiv  Detail & Related papers  (2020-07-31T03:48:41Z)
• Grounded Situation Recognition [56.18102368133022]
  We introduce Grounded Situation Recognition (GSR), a task that requires producing structured semantic summaries of images. GSR presents important technical challenges: identifying semantic saliency, categorizing and localizing a large and diverse set of entities. We show initial findings on three exciting future directions enabled by our models: conditional querying, visual chaining, and grounded semantic aware image retrieval.
  arXiv  Detail & Related papers  (2020-03-26T17:57:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.