ASTRA: Enhancing Multi-Subject Generation with Retrieval-Augmented Pose Guidance and Disentangled Position Embedding

• URL: http://arxiv.org/abs/2604.13938v1
• Date: Wed, 15 Apr 2026 14:50:05 GMT
• Title: ASTRA: Enhancing Multi-Subject Generation with Retrieval-Augmented Pose Guidance and Disentangled Position Embedding
• Abstract summary: ASTRA is a novel framework that architecturally disentangles subject appearance from pose structure.<n>ASTRA achieves a new state-of-the-art in pose adherence, while maintaining high identity fidelity and text alignment in DreamBench.
• Score: 1.4145309506833208
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Subject-driven image generation has shown great success in creating personalized content, but its capabilities are largely confined to single subjects in common poses. Current approaches face a fundamental conflict when handling multiple subjects with complex, distinct actions: preserving individual identities while enforcing precise pose structures. This challenge often leads to identity fusion and pose distortion, as appearance and structure signals become entangled within the model's architecture. To resolve this conflict, we introduce ASTRA(Adaptive Synthesis through Targeted Retrieval Augmentation), a novel framework that architecturally disentangles subject appearance from pose structure within a unified Diffusion Transformer. ASTRA achieves this through a dual-pronged strategy. It first employs a Retrieval-Augmented Pose (RAG-Pose) pipeline to provide a clean, explicit structural prior from a curated database. Then, its core generative model learns to process these dual visual conditions using our Enhanced Universal Rotary Position Embedding (EURoPE), an asymmetric encoding mechanism that decouples identity tokens from spatial locations while binding pose tokens to the canvas. Concurrently, a Disentangled Semantic Modulation (DSM) adapter offloads the identity preservation task into the text conditioning stream. Extensive experiments demonstrate that our integrated approach achieves superior disentanglement. On our designed COCO-based complex pose benchmark, ASTRA achieves a new state-of-the-art in pose adherence, while maintaining high identity fidelity and text alignment in DreamBench.

Related papers

• DivRL: Disentangled Self-Similarity Rewards for Diverse Subject-Driven Generation [52.23323703015865]
  We propose a post-training framework called DivRL that jointly optimize identity consistency and structural diversity simultaneously.<n>We introduce a Negative Self-Similarity Measure (nSSM) to quantify structural diversity, and Visual Semantic Matching (VSM) to evaluate identity consistency.
  arXiv  Detail & Related papers  (2026-06-22T21:22:42Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers [93.3976834364707]
  We propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps.<n>By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence.
  arXiv  Detail & Related papers  (2026-05-16T12:01:04Z)
• A Unified Conditional Flow for Motion Generation, Editing, and Intra-Structural Retargeting [26.68081874066983]
  We present a unifying perspective where both tasks are cast as instances of conditional transport within a single generative framework.<n>We implement this vision via a rectified-flow motion model jointly conditioned and target skeletal structures.<n>Experiments on SnapMoGen and a multi-character Mixamo subset show that a single trained model supports text-to-motion generation, zero-shot editing, and zero-shot intra-structural.
  arXiv  Detail & Related papers  (2026-04-15T02:53:07Z)
• Directing the Narrative: A Finetuning Method for Controlling Coherence and Style in Story Generation [2.9255420892138306]
  We propose a two-stage framework designed for robust and consistent story generation.<n>First, we introduce Group-Shared Attention (GSA), a mechanism that fosters intrinsic consistency.<n>Second, we leverage Direct Preference Optimization (DPO) to align generated outputs with human aesthetic and narrative standards.
  arXiv  Detail & Related papers  (2026-03-18T02:43:02Z)
• AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation [55.94507360511886]
  We propose AnyCrowd, a Transformer-based video generation framework capable of scaling to an arbitrary number of characters.<n>Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity entanglement.<n>Building on this disentangled representation, we further propose Tri-Stage Decoupled Attention (TSDA) to bind identities to driving poses by decomposing self-attention into: (i) instance-aware foreground attention, (ii) background-centric interaction, and (iii) global-background coordination.
  arXiv  Detail & Related papers  (2026-03-16T15:25:04Z)
• Component-Aware Sketch-to-Image Generation Using Self-Attention Encoding and Coordinate-Preserving Fusion [2.510998372750843]
  Translating freehand sketches into photorealistic images remains a fundamental challenge in image synthesis.<n>Existing approaches, including GAN-based and diffusion-based models, often struggle to reconstruct fine-grained details, maintain spatial alignment, or adapt across different sketch domains.<n>We propose a component-aware, self-refining framework for sketch-to-image generation that addresses these challenges through a novel two-stage architecture.
  arXiv  Detail & Related papers  (2026-03-10T10:39:24Z)
• PICS: Pairwise Image Compositing with Spatial Interactions [20.306534705755613]
  PICS is a self-supervised composition-by-decomposition paradigm that composes objects in parallel.<n>An Interaction Transformer employs mask-guided Mixture-of-Experts to route background, exclusive, and overlap regions.<n>To further enhance robustness to geometric variations, we incorporate geometry-aware augmentations covering both out-of-plane and in-plane pose changes.
  arXiv  Detail & Related papers  (2026-03-06T20:48:21Z)
• Structural Action Transformer for 3D Dexterous Manipulation [80.07649565189035]
  Cross-embodiment skill transfer is a challenge for high-DoF robotic hands.<n>Existing methods, often relying on 2D observations and temporal-centric action representation, struggle to capture 3D spatial relations.<n>This paper proposes a new 3D dexterous manipulation policy that challenges this paradigm by introducing a structural-centric perspective.
  arXiv  Detail & Related papers  (2026-03-04T11:38:12Z)
• DreamID-V:Bridging the Image-to-Video Gap for High-Fidelity Face Swapping via Diffusion Transformer [21.788582116033684]
  Video Face Swapping (VFS) requires seamlessly injecting a source identity into a target video.<n>Existing methods struggle to maintain identity similarity and attribute preservation while preserving temporal consistency.<n>We propose a comprehensive framework to seamlessly transfer the superiority of Image Face Swapping to the video domain.
  arXiv  Detail & Related papers  (2026-01-04T08:07:11Z)
• Towards Robust and Realistic Human Pose Estimation via WiFi Signals [85.60557095666934]
  WiFi-based human pose estimation is a challenging task that bridges discrete and subtle WiFi signals to human skeletons.<n>This paper revisits this problem and reveals two critical yet overlooked issues: 1) cross-domain gap, i.e., due to significant variations between source-target domain pose distributions; and 2) structural fidelity gap, i.e., predicted skeletal poses manifest distorted topology.<n>This paper fills these gaps by reformulating the task into a novel two-phase framework dubbed DT-Pose: Domain-consistent representation learning and Topology-constrained Pose decoding.
  arXiv  Detail & Related papers  (2025-01-16T09:38:22Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)
• Joint Disentangling and Adaptation for Cross-Domain Person Re-Identification [88.79480792084995]
  We propose a joint learning framework that disentangles id-related/unrelated features and enforces adaptation to work on the id-related feature space exclusively. Our model involves a disentangling module that encodes cross-domain images into a shared appearance space and two separate structure spaces, and an adaptation module that performs adversarial alignment and self-training on the shared appearance space.
  arXiv  Detail & Related papers  (2020-07-20T17:57:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.