RAVA: Retrieval-Augmented Viewpoint Alignment for Subject-Driven Image Generation

• URL: http://arxiv.org/abs/2606.17619v1
• Date: Tue, 16 Jun 2026 07:25:08 GMT
• Title: RAVA: Retrieval-Augmented Viewpoint Alignment for Subject-Driven Image Generation
• Abstract summary: Cross-subject viewpoint alignment is a challenge in reference-driven image generation.<n>We propose RAVA, a retrieval-augmented framework that supplies explicit geometric evidence before generation.<n> RAVA consistently outperforms zero-shot baselines on cross-subject generation.
• Score: 21.751544721133005
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reference-driven image generation has made rapid progress on identity preservation, but reliable viewpoint control across different subjects remains poorly understood. The difficulty is not merely generating a new image of the target subject: the model must infer the implicit viewpoint of one subject and transfer it to another subject using only image-level evidence, without camera poses, depth, or ray-based conditions. In this setting, existing generators conditioned on multiple image references often rely on spurious semantic correlations, which lead to viewpoint drift, part-level structural mismatches, and missing or unsupported target-specific content. We formulate this challenge as cross-subject viewpoint alignment and propose RAVA, a retrieval-augmented framework that supplies explicit geometric evidence before generation. RAVA first learns a cross-instance viewpoint embedding that retrieves target-subject images aligned with the anchor viewpoint, then applies a LogDet-based subset selection strategy to retain a compact reference set that is both view-consistent and structurally complementary. The selected references are finally consumed by a fine-tuned multi-reference image generator. Experiments show that generic semantic embeddings are nearly random for this task, while the proposed retriever substantially improves viewpoint retrieval quality. On cross-subject generation, RAVA consistently outperforms zero-shot baselines and stronger retrieval alternatives under the same generation backbone. These results indicate that cross-subject viewpoint alignment benefits from retrieval-augmented geometric grounding rather than relying on end-to-end generation alone.

Related papers

• Beyond Semantic Search: Towards Referential Anchoring in Composed Image Retrieval [56.72697928941446]
  Composed Image Retrieval (CIR) has demonstrated significant potential by enabling flexible multimodal queries that combine a reference image and modification text.<n>We propose Object-Anchored Composed Image Retrieval (OACIR), a novel fine-grained retrieval task that mandates strict instance-level consistency.
  arXiv  Detail & Related papers  (2026-04-07T03:43:01Z)
• Self-Corrected Image Generation with Explainable Latent Rewards [55.29175717238288]
  We propose xLARD, a self-correcting framework that guides generation through Explainable LAtent RewarDs.<n>xLARD introduces a lightweight corrector that refines latent representations based on structured feedback from model-generated references.<n> Experiments show that xLARD improves semantic alignment and visual fidelity while maintaining generative priors.
  arXiv  Detail & Related papers  (2026-03-26T02:59:35Z)
• SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation [62.55421542903781]
  We present textbfSpatialReward, a verifiable reward model explicitly designed to evaluate spatial layouts in generated images.<n>Experiments on Stable Diffusion and FLUX show that incorporating SpatialReward into RL training consistently improves spatial consistency and overall generation quality.
  arXiv  Detail & Related papers  (2026-03-23T17:26:35Z)
• Cross-modal RAG: Sub-dimensional Text-to-Image Retrieval-Augmented Generation [16.411920699543774]
  Cross-modal RAG is a novel framework that decomposes both queries and images into sub-dimensional components.<n>Our method introduces a hybrid retrieval strategy - combining a sub-dimensional sparse retriever with a dense retriever.<n>Experiments on MS-COCO, Flickr30K, WikiArt, CUB, and ImageNet-LT demonstrate that Cross-modal RAG significantly outperforms existing baselines in the retrieval.
  arXiv  Detail & Related papers  (2025-05-28T04:09:49Z)
• RefPose: Leveraging Reference Geometric Correspondences for Accurate 6D Pose Estimation of Unseen Objects [21.047434860977454]
  RefPose is an innovative approach to object pose estimation that leverages a reference image and geometric correspondence as guidance.<n>We show that RefPose achieves state-of-the-art results while maintaining a competitive runtime.
  arXiv  Detail & Related papers  (2025-05-16T04:17:58Z)
• UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image [86.7128543480229]
  Unseen object pose estimation methods often rely on CAD models or multiple reference views.<n>To simplify reference acquisition, we aim to estimate the unseen object's pose through a single unposed RGB-D reference image.<n>We present a novel approach and benchmark, termed UNOPose, for unseen one-reference-based object pose estimation.
  arXiv  Detail & Related papers  (2024-11-25T05:36:00Z)
• RetriBooru: Leakage-Free Retrieval of Conditions from Reference Images for Subject-Driven Generation [30.143033020296183]
  Diffusion-based methods have demonstrated capabilities in generating a diverse array of high-quality images. We propose a multi-level, same-identity dataset RetriBooru, which groups anime characters by both face and cloth identities. We introduce a new concept composition task, where the conditioning encoder learns to retrieve different concepts from several reference images.
  arXiv  Detail & Related papers  (2023-12-05T06:04:16Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)
• Object-Centric Image Generation from Layouts [93.10217725729468]
  We develop a layout-to-image-generation method to generate complex scenes with multiple objects. Our method learns representations of the spatial relationships between objects in the scene, which lead to our model's improved layout-fidelity. We introduce SceneFID, an object-centric adaptation of the popular Fr'echet Inception Distance metric, that is better suited for multi-object images.
  arXiv  Detail & Related papers  (2020-03-16T21:40:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.