PoseAdapter: Dual-Stream 2.5D Controllable Image Generation for Complex Multi-Object Scenes

• URL: http://arxiv.org/abs/2608.15583v1
• Date: Sun, 16 Aug 2026 07:18:58 GMT
• Title: PoseAdapter: Dual-Stream 2.5D Controllable Image Generation for Complex Multi-Object Scenes
• Abstract summary: PoseAdapter is a framework for high-fidelity 2.5D controllable image generation.<n>It establishes precise spatial-angular anchors using individual object captions, 2D bounding boxes, and 3D angles.<n>It outperforms state-of-the-art baselines in spatial accuracy, orientational precision, and multi-object visual fidelity.
• Score: 32.05027130743237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Text-to-Image (T2I) diffusion models have achieved remarkable success, precise spatial and orientational control in multi-object scenes remains a persistent challenge. Existing methods either rely on computationally expensive dense 3D maps or suffer from severe attribute leakage and "cut-and-paste" artifacts. To address these limitations, we propose PoseAdapter, a lightweight framework for high-fidelity 2.5D controllable image generation. Instead of dense spatial maps, it establishes precise spatial-angular anchors using an efficient condition layout: individual object captions, 2D bounding boxes, and 3D angles. To resolve the generative trade-off between strict instance isolation and global coherence, we introduce a Context-Aware Dual-Stream Representation. By injecting local object tokens and relation-enriched scene tokens into the visual stream of modern MM-DiT architectures via parallel masked and unmasked pathways, PoseAdapter eliminates attribute leakage while preserving natural inter-object relationships and scene-level coherence. To support this paradigm, we construct OrientLayout, a high-quality dataset featuring standardized 2.5D annotations and instance-level decoupled semantics. Extensive experiments demonstrate that PoseAdapter outperforms state-of-the-art baselines in spatial accuracy, orientational precision, and multi-object visual fidelity. Code and dataset will be available at https://github.com/cyf23/PoseAdapter.

Related papers

• Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization [21.013795754583697]
  Cross-view object geo-localization (CVOGL) aims to locate a target object from a query view within a geo-tagged reference image (e.g., satellite)<n>Existing approaches heavily rely on 2D appearance matching and are constrained by limited datasets lacking geometric metadata.<n>We first introduce dataset, a large-scale, high-fidelity building dataset comprising over 220,000 ground-satellite and drone-satellite pairs.<n>We propose a novel single-stage Geometry-Aware Geo-localization framework (GAGeo), built upon the permutation-equivariant 3D foundation model $
  arXiv  Detail & Related papers  (2026-06-29T17:19:49Z)
• Pose-ICL: 3D-Aware In-Context Learning for Pose-Controllable Subject Customization [9.596779036612409]
  Pose-ICL is a tuning-free framework that adapts to new subjects through multiple paired image-pose references.<n>Its core mechanism,Surface-Anchored Position Embedding (SAPE), equips the model with explicit 3D awareness by anchoring image tokens to the surface coordinates of a volumetric bounding box.<n>Extensive evaluations on both 3D assets and real-world subjects demonstrate that Pose-ICL significantly outperforms current methods in both pose accuracy and identity consistency.
  arXiv  Detail & Related papers  (2026-06-09T14:13:40Z)
• TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation [1.709500663078044]
  Trio is a pose-driven framework built upon the SD3.5M architecture.<n>It achieves state-of-the-art performance across challenging benchmarks, including Human-Art, CrowdPose, and OCHuman.<n> Notably, it attains an AP of $64.33$ on Human-Art, representing a $30%$ improvement over prior arts.
  arXiv  Detail & Related papers  (2026-06-05T08:54:28Z)
• Direct 3D-Aware Object Insertion via Decomposed Visual Proxies [101.76006356727015]
  DIRECT (Decomposed Injection for Reference Composition and Target-integration) is a novel framework that integrates interactive pose manipulation with high-fidelity 2D image synthesis.<n>Our method decomposes the insertion conditions into three complementary components: appearance guidance capturing visual details from the reference object, geometry guidance derived from the user-adjusted 3D proxy, and context guidance from the target background.<n>Experiments show that DIRECT outperforms previous methods in both geometric controllability and visual quality.
  arXiv  Detail & Related papers  (2026-06-04T18:00:34Z)
• ObjEmbed: Towards Universal Multimodal Object Embeddings [74.39703419628829]
  We present Embed, a novel individual object embedding model.<n>It decomposes the input image into multiple regional embeddings, each corresponding to an individual object, along with global embeddings.<n>It supports a wide range of visual understanding tasks like visual retrieval, local image retrieval, and global image retrieval.
  arXiv  Detail & Related papers  (2026-02-02T07:38:45Z)
• RoamScene3D: Immersive Text-to-3D Scene Generation via Adaptive Object-aware Roaming [79.81527946524098]
  RoamScene3D is a novel framework that bridges the gap between semantic guidance and spatial generation.<n>We employ a vision-language model (VLM) to construct a scene graph that encodes object relations.<n>To mitigate the limitations of static 2D priors, we introduce a Motion-Injected Inpainting model that is fine-tuned on a synthetic panoramic dataset.
  arXiv  Detail & Related papers  (2026-01-27T10:10:55Z)
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction [82.53307702809606]
  Humans naturally perceive the geometric structure and semantic content of a 3D world as intertwined dimensions.<n>We propose InstanceGrounded Geometry Transformer (IGGT) to unify the knowledge for both spatial reconstruction and instance-level contextual understanding.
  arXiv  Detail & Related papers  (2025-10-26T14:57:44Z)
• GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation [81.0871900167463]
  We introduce GeoSAM2, a prompt-controllable framework for 3D part segmentation.<n>Given a textureless object, we render normal and point maps from predefined viewpoints.<n>We accept simple 2D prompts - clicks or boxes - to guide part selection.<n>The predicted masks are back-projected to the object and aggregated across views.
  arXiv  Detail & Related papers  (2025-08-19T17:58:51Z)
• SCORP: Scene-Consistent Object Refinement via Proxy Generation and Tuning [46.441761732998536]
  We introduce Scene-Consistent Object Refinement via Proxy Generation and Tuning (SCORP)<n>SCORP is a novel 3D enhancement framework that leverages 3D generative priors to recover fine-grained object geometry and appearance under missing views.<n>It achieves consistent gains over recent state-of-the-art baselines on both novel view synthesis and geometry completion tasks.
  arXiv  Detail & Related papers  (2025-06-30T13:26:21Z)
• HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation [50.206100327643284]
  HiScene is a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation.<n>We generate 3D content that aligns with 2D representations while maintaining compositional structure.
  arXiv  Detail & Related papers  (2025-04-17T16:33:39Z)
• A3D: Does Diffusion Dream about 3D Alignment? [73.97853402817405]
  We tackle the problem of text-driven 3D generation from a geometry alignment perspective.<n>Given a set of text prompts, we aim to generate a collection of objects with semantically corresponding parts aligned across them.<n>We propose to embed these objects into a common latent space and optimize the continuous transitions between these objects.
  arXiv  Detail & Related papers  (2024-06-21T09:49:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.