TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation

• URL: http://arxiv.org/abs/2606.07053v1
• Date: Fri, 05 Jun 2026 08:54:28 GMT
• Title: TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation
• Abstract summary: Trio is a pose-driven framework built upon the SD3.5M architecture.<n>It achieves state-of-the-art performance across challenging benchmarks, including Human-Art, CrowdPose, and OCHuman.<n> Notably, it attains an AP of $64.33$ on Human-Art, representing a $30%$ improvement over prior arts.
• Score: 1.709500663078044
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pose-guided text-to-image generation often suffers from limb distortions and feature crosstalk in complex multi-person scenarios. While existing UNet-based adapters struggle with long-range spatial dependencies, emerging Multimodal Diffusion Transformers (MM-DiTs) offer superior global modeling. However, naive signal concatenation in MM-DiTs severely disrupts pre-trained latent distributions. To address this, we propose TrioPose, a native pose-driven framework built upon the SD3.5M architecture. Specifically, we introduce a Triple-Stream Pose-Aware DiT (TSPA-DiT) that treats pose as an independent modality. It employs layer-wise activation and zero-initialized dual-residual injection to smoothly enforce geometric constraints while preserving pre-trained latent stability. To resolve severe multi-instance occlusions, we design a Learnable Relational Bias Mask that categorizes topological connectivity into fine-grained physical states, mapping them into continuous attention soft constraints to effectively decouple inter-instance interference. Furthermore, a Pose-Guided Spatial Loss Weighting strategy modulates the native diffusion objective using heatmap-derived error maps, focusing anatomical supervision strictly on distortion-prone regions. Extensive experiments demonstrate that TrioPose achieves state-of-the-art performance across challenging benchmarks, including Human-Art, CrowdPose, and OCHuman. Notably, it attains an AP of $64.33$ on Human-Art, representing a $30\%$ improvement over prior arts, while setting new standards for visual fidelity and text-image semantic alignment in complex multi-human generation.

Related papers

• Inclusive Interactive Collisions for Multi-View Consistent Compositional 3D Generation [25.750694245523217]
  We propose I2C-3D, a novel optimization-based method to generate multi-view consistent compositional 3D assets with reasonable interactions.<n>Specifically, we propose an Inclusive Interactive Collisions strategy to guide Gaussian primitives appearing in reasonable interaction regions naturally.<n>To enhance multi-view consistency, Multi-View Adaptive Score Distillation Sampling is devised to distill multi-view consistency prior and layout prior from pre-trained diffusion model.
  arXiv  Detail & Related papers  (2026-06-23T06:48:22Z)
• Physically-guided Image Generation for Multi-Projection Mapping [13.227477542441918]
  Projection Mapping (PM) enables seamless superimposition of digital content onto real-world 3D objects.<n>This paper formalizes two application-level generative paradigms: the cooperative paradigm and the adversarial paradigm.<n>We propose ConPhyG, a unified controllable generative multi-projection mapping framework.
  arXiv  Detail & Related papers  (2026-06-21T12:37:20Z)
• DisPOSE: Projected Polystochastic Diffusion for Self-Supervised Multi-View 3D Human Pose Estimation [62.92906479014098]
  DisPOSE is a self-supervised framework that approximates the inherently discrete multi-view person-assignment problem.<n>By employing differentiable Sinkhorn projections during denoising, our model learns to guide solutions toward valid and feasible assignments.<n>The proposed approach outperforms current state-of-the-art self-supervised methods on standard datasets.
  arXiv  Detail & Related papers  (2026-06-05T16:14:39Z)
• SpatialFusion: Endowing Unified Image Generation with Intrinsic 3D Geometric Awareness [62.38746300197673]
  We propose a novel framework that internalizes 3D geometric awareness into unified image generation models.<n>Specifically, we first employ a Mixture-of-Transformers (MoT) architecture to augment the MLLM with a parallel spatial transformer.<n>By sharing self-attention with the MLLM, the spatial transformer learns to derive metric-depth maps of target images from rich semantic contexts.<n>These explicit geometric scaffolds are then injected into the diffusion backbone through a specialized depth adapter.
  arXiv  Detail & Related papers  (2026-04-29T06:46:59Z)
• Contrastive Multi-Modal Hypergraph Reasoning for 3D Crowd Mesh Recovery [12.116457701406047]
  Multi-person 3D reconstruction is pivotal for real-world interaction analysis.<n>Current approaches rely on single-modality inputs, which inherently lack geometric guidance.<n>We propose Contrastive Multi-modal Hypergraph Reasoning to synergize semantic, geometric, and pose cues for crowd reconstruction.
  arXiv  Detail & Related papers  (2026-04-01T09:39:01Z)
• InpaintHuman: Reconstructing Occluded Humans with Multi-Scale UV Mapping and Identity-Preserving Diffusion Inpainting [64.42884719282323]
  InpaintHuman is a novel method for generating high-fidelity, complete, and animatable avatars from occluded monocular videos.<n>Our approach employs direct pixel-level supervision to ensure identity fidelity.
  arXiv  Detail & Related papers  (2026-01-05T13:26:02Z)
• HBSplat: Robust Sparse-View Gaussian Reconstruction with Hybrid-Loss Guided Depth and Bidirectional Warping [11.035994094874141]
  HBSplat is a framework that seamlessly integrates robust structural cues, virtual view constraints, and occluded region completion.<n> HBSplat sets a new state-of-the-art, achieving up to 21.13 dB PSNR and 0.189 LPIPS, while maintaining real-time inference.
  arXiv  Detail & Related papers  (2025-09-29T15:03:31Z)
• Pose-RFT: Enhancing MLLMs for 3D Pose Generation via Hybrid Action Reinforcement Fine-Tuning [36.35364256178837]
  Pose-RFT is a reinforcement fine-tuning framework tailored for 3D human pose generation in MLLMs.<n>We introduce HyGRPO, a hybrid reinforcement learning algorithm that performs group-wise reward normalization over sampled responses.<n>Experiments on multiple pose generation benchmarks demonstrate that Pose-RFT significantly improves performance over existing pose-specific MLLMs.
  arXiv  Detail & Related papers  (2025-08-11T09:44:58Z)
• GCA-3D: Towards Generalized and Consistent Domain Adaptation of 3D Generators [24.67369444661137]
  GCA-3D is a generalized and consistent 3D domain adaptation method without the intricate pipeline of data generation.<n>We introduce multi-modal depth-aware score distillation sampling loss to efficiently adapt 3D generative models in a non-adversarial manner.<n>Experiments demonstrate that GCA-3D outperforms previous methods in terms of efficiency, generalization, pose accuracy, and identity consistency.
  arXiv  Detail & Related papers  (2024-12-20T02:13:11Z)
• GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision [49.839374549646884]
  This paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception.<n>Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone.
  arXiv  Detail & Related papers  (2024-05-17T07:31:20Z)
• Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation [63.199549837604444]
  3D human pose estimation approaches leverage different forms of strong (2D/3D pose) or weak (multi-view or depth) paired supervision. We cast 3D pose learning as a self-supervised adaptation problem that aims to transfer the task knowledge from a labeled source domain to a completely unpaired target. We evaluate different self-adaptation settings and demonstrate state-of-the-art 3D human pose estimation performance on standard benchmarks.
  arXiv  Detail & Related papers  (2022-04-05T03:52:57Z)
• Unsupervised Geodesic-preserved Generative Adversarial Networks for Unconstrained 3D Pose Transfer [84.04540436494011]
  We present an unsupervised approach to conduct the pose transfer between any arbitrated given 3D meshes. Specifically, a novel Intrinsic-Extrinsic Preserved Generative Adrative Network (IEP-GAN) is presented for both intrinsic (i.e., shape) and extrinsic (i.e., pose) information preservation. Our proposed model produces better results and is substantially more efficient compared to recent state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-17T09:08:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.