Text-Image Conditioned 3D Generation

• URL: http://arxiv.org/abs/2603.21295v1
• Date: Sun, 22 Mar 2026 15:36:16 GMT
• Title: Text-Image Conditioned 3D Generation
• Abstract summary: TIGON is a minimalist dual-branch baseline with separate image- and text-conditioned backbones and lightweight cross-modal fusion.<n>Our diagnostic study shows that even simple late fusion of text- and image-conditioned predictions outperforms single-modality models.<n>Extensive experiments show that text-image conditioning consistently improves over single-modality methods.
• Score: 71.98375600100856
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: High-quality 3D assets are essential for VR/AR, industrial design, and entertainment, motivating growing interest in generative models that create 3D content from user prompts. Most existing 3D generators, however, rely on a single conditioning modality: image-conditioned models achieve high visual fidelity by exploiting pixel-aligned cues but suffer from viewpoint bias when the input view is limited or ambiguous, while text-conditioned models provide broad semantic guidance yet lack low-level visual detail. This limits how users can express intent and raises a natural question: can these two modalities be combined for more flexible and faithful 3D generation? Our diagnostic study shows that even simple late fusion of text- and image-conditioned predictions outperforms single-modality models, revealing strong cross-modal complementarity. We therefore formalize Text-Image Conditioned 3D Generation, which requires joint reasoning over a visual exemplar and a textual specification. To address this task, we introduce TIGON, a minimalist dual-branch baseline with separate image- and text-conditioned backbones and lightweight cross-modal fusion. Extensive experiments show that text-image conditioning consistently improves over single-modality methods, highlighting complementary vision-language guidance as a promising direction for future 3D generation research. Project page: https://jumpat.github.io/tigon-page

Related papers

• Omni123: Exploring 3D Native Foundation Models with Limited 3D Data by Unifying Text to 2D and 3D Generation [15.607138815246387]
  We present Omni123, a 3D-native foundation model that unifies text-to-2D and text-to-3D generation within a single autoregressive framework.<n> Experiments show that Omni123 significantly improves text-guided 3D generation and editing, demonstrating a scalable path toward multimodal 3D world models.
  arXiv  Detail & Related papers  (2026-04-02T17:29:38Z)
• Dual Diffusion Models for Multi-modal Guided 3D Avatar Generation [19.94446175293186]
  We present PromptAvatar, a framework for generating high-fidelity 3D avatars from text or image prompts.<n>It learns the direct mapping from multi-modal prompts to 3D representations, successfully generating high-fidelity, shading-free 3D avatars in under 10 seconds.<n>Our method significantly outperforms existing state-of-the-art approaches in generation quality, fine-grained detail alignment, and computational efficiency.
  arXiv  Detail & Related papers  (2026-03-04T17:24:28Z)
• Controllable 3D Object Generation with Single Image Prompt [2.4622211579286133]
  3D object generation tasks are one of the fastest-growing segments in computer vision.<n>Text-to-image generative models employ textual inversion to learn concepts or styles of target object in the pseudo text prompt embedding space.<n>We propose two innovative methods: (1) using an off-the-shelf image adapter that generates 3D objects without textual inversion, offering enhanced control over conditions such as depth, pose, and text; and (2) a depth conditioned warmup strategy to enhance 3D consistency.
  arXiv  Detail & Related papers  (2025-11-27T08:03:56Z)
• Any-to-3D Generation via Hybrid Diffusion Supervision [67.54197818071464]
  XBind is a unified framework for any-to-3D generation using cross-modal pre-alignment techniques. XBind integrates an multimodal-aligned encoder with pre-trained diffusion models to generate 3D objects from any modalities.
  arXiv  Detail & Related papers  (2024-11-22T03:52:37Z)
• SeMv-3D: Towards Concurrency of Semantic and Multi-view Consistency in General Text-to-3D Generation [122.47961178994456]
  SeMv-3D is a novel framework that jointly enhances semantic alignment and multi-view consistency in GT23D generation.<n>At its core, we introduce Triplane Prior Learning (TPL), which effectively learns triplane priors.<n>We also present Prior-based Semantic Aligning in Triplanes (SAT), which enables consistent any-view synthesis.
  arXiv  Detail & Related papers  (2024-10-10T07:02:06Z)
• Phidias: A Generative Model for Creating 3D Content from Text, Image, and 3D Conditions with Reference-Augmented Diffusion [59.00571588016896]
  In 3D modeling, designers often use an existing 3D model as a reference to create new ones. This practice has inspired the development of Phidias, a novel generative model that uses diffusion for reference-augmented 3D generation.
  arXiv  Detail & Related papers  (2024-09-17T17:59:33Z)
• Guide3D: Create 3D Avatars from Text and Image Guidance [55.71306021041785]
  Guide3D is a text-and-image-guided generative model for 3D avatar generation based on diffusion models. Our framework produces topologically and structurally correct geometry and high-resolution textures.
  arXiv  Detail & Related papers  (2023-08-18T17:55:47Z)
• Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation [72.94143731623117]
  Existing methods simply align 3D representations with single-view 2D images and coarse-grained parent category text. Insufficient synergy neglects the idea that a robust 3D representation should align with the joint vision-language space. We propose a multi-view joint modality modeling approach, termed JM3D, to obtain a unified representation for point cloud, text, and image.
  arXiv  Detail & Related papers  (2023-08-06T01:11:40Z)
• DreamBooth3D: Subject-Driven Text-to-3D Generation [43.14506066034495]
  We present DreamBooth3D, an approach to personalize text-to-3D generative models from as few as 3-6 casually captured images of a subject. We find that naively combining these methods fails to yield satisfactory subject-specific 3D assets due to personalized text-to-image models overfitting to the input viewpoints of the subject. Our method can produce high-quality, subject-specific 3D assets with text-driven modifications such as novel poses, colors and attributes that are not seen in any of the input images of the subject.
  arXiv  Detail & Related papers  (2023-03-23T17:59:00Z)
• On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization [89.94078728495423]
  We show that recent advances in each modality, CLIP image representations and scaling of language models, do not consistently improve multimodal self-rationalization of tasks with multimodal inputs. Our findings call for a backbone modelling approach that can be built on to advance text generation from images and text beyond image captioning.
  arXiv  Detail & Related papers  (2022-05-24T00:52:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.