Tree-of-Thoughts Reasoning for Text-to-Image In-Context Learning

• URL: http://arxiv.org/abs/2607.07117v1
• Date: Wed, 08 Jul 2026 07:58:48 GMT
• Title: Tree-of-Thoughts Reasoning for Text-to-Image In-Context Learning
• Abstract summary: In text-to-image in-context learning (T2I-ICL), a model has to infer a latent compositional pattern from fewshot demonstrations for generating a query image.<n>Recent studies show that state-of-the-art multimodal large language models struggle with this setting.<n>We propose a Tree-of-Thoughts (ToT) reasoning framework for T2I-ICL that introduces a multi-stage reasoning and selection layer.
• Score: 45.88028371034407
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In text-to-image in-context learning (T2I-ICL), a model has to infer a latent compositional pattern from fewshot demonstrations for generating a query image. Recent studies show that state-of-the-art multimodal large language models struggle with this setting, particularly due to limited compositional reasoning and sensitivity to prompt construction. In this work, we propose a Tree-of-Thoughts (ToT) reasoning framework for T2I-ICL that introduces a multi-stage reasoning and selection layer that generates, evaluates, and selects among multiple candidate hypotheses before constructing the final prompt for image synthesis. By exploring alternative reasoning branches and selecting a coherent interpretation, the proposed approach mitigates prompt ambiguity and compositional errors. We implement the proposed approach in a complete ToT-T2IICL inference pipeline and evaluate it on the CoBSAT benchmark. Both qualitative and quantitative results show that structured multi-branch reasoning leads to more consistent and semantically aligned image generation compared to baseline and Chain-of-Thought prompting strategies, without any additional training or fine-tuning.

Related papers

• Exploring Multi-Modal Large Language Models and Two-Stage Fine-Tuning for Fashion Image Retrieval [3.1033110046835457]
  In the fashion domain, this task requires understanding subtle variations such as color, pattern, and texture.<n>Existing approaches face limitations due to scarce annotated data and simplistic negative sampling.<n>We propose a novel framework that integrates a multi-grained large language model (LLaVA) to generate attribute-aware triplets.
  arXiv  Detail & Related papers  (2026-06-18T01:19:32Z)
• iDiff: Interpretable Difference-aware Framework for Pairwise Image Quality Assessment [10.381055159281823]
  We propose iDiff, an Interpretable Difference-aware framework for pairwise image quality assessment.<n>Our method achieves first place in the NTIRE 2026 RAIM challenge.
  arXiv  Detail & Related papers  (2026-05-19T08:24:00Z)
• Constructing and Interpreting Digital Twin Representations for Visual Reasoning via Reinforcement Learning [9.529907786822115]
  We propose DT-R1, a reinforcement learning framework that trains large language models to construct digital twin representations of complex visual inputs.<n>We show that DT-R1 consistently achieves improvements over state-of-the-art task-specific models.
  arXiv  Detail & Related papers  (2025-11-15T21:57:25Z)
• ThinkFake: Reasoning in Multimodal Large Language Models for AI-Generated Image Detection [51.93101033997245]
  Increasing realism of AI-generated images has raised serious concerns about misinformation and privacy violations.<n>We propose ThinkFake, a novel reasoning-based and generalizable framework for AI-generated image detection.<n>We show that ThinkFake outperforms state-of-the-art methods on the GenImage benchmark and demonstrates strong zero-shot generalization on the challenging LOKI benchmark.
  arXiv  Detail & Related papers  (2025-09-24T07:34:09Z)
• Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play? [63.66192651248858]
  T2I-CoReBench is a comprehensive and complex benchmark that evaluates both composition and reasoning capabilities of T2I models.<n>To increase complexity, driven by the inherent real-world complexities, we curate each prompt with higher compositional density for composition.<n>In statistics, our benchmark comprises 1,080 challenging prompts and around 13,500 checklist questions.
  arXiv  Detail & Related papers  (2025-09-03T17:58:12Z)
• Can Generated Images Serve as a Viable Modality for Text-Centric Multimodal Learning? [3.966028515034415]
  This work systematically investigates whether images generated on-the-fly by Text-to-Image (T2I) models can serve as a valuable complementary modality for text-centric tasks.
  arXiv  Detail & Related papers  (2025-06-21T07:32:09Z)
• RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning [88.14234949860105]
  RePrompt is a novel reprompting framework that introduces explicit reasoning into the prompt enhancement process via reinforcement learning.<n>Our approach enables end-to-end training without human-annotated data.
  arXiv  Detail & Related papers  (2025-05-23T06:44:26Z)
• R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization [26.757458496178437]
  We introduce R1-Onevision, a multimodal reasoning model designed to bridge the gap between visual perception and deep reasoning.<n>We construct the R1-Onevision dataset which provides detailed, step-by-step multimodal reasoning annotations across diverse domains.<n>We further develop the R1-Onevision model through supervised fine-tuning and reinforcement learning to cultivate advanced reasoning.<n> Experimental results show that R1-Onevision achieves state-of-the-art performance, outperforming models such as GPT-4o and Qwen2.5-VL.
  arXiv  Detail & Related papers  (2025-03-13T17:56:05Z)
• On the Diagram of Thought [20.805936414171892]
  Large Language Models (LLMs) excel at many tasks but often falter on complex problems that require structured, multi-step reasoning.<n>We introduce the Diagram of Thought (DoT), a new framework that enables a single LLM to build and navigate a mental map of its reasoning.
  arXiv  Detail & Related papers  (2024-09-16T07:01:41Z)
• Multimodal Relation Extraction with Cross-Modal Retrieval and Synthesis [89.04041100520881]
  This research proposes to retrieve textual and visual evidence based on the object, sentence, and whole image. We develop a novel approach to synthesize the object-level, image-level, and sentence-level information for better reasoning between the same and different modalities.
  arXiv  Detail & Related papers  (2023-05-25T15:26:13Z)
• Linguistic Structure Guided Context Modeling for Referring Image Segmentation [61.701577239317785]
  We propose a "gather-propagate-distribute" scheme to model multimodal context by cross-modal interaction. Our LSCM module builds a Dependency Parsing Tree Word Graph (DPT-WG) which guides all the words to include valid multimodal context of the sentence.
  arXiv  Detail & Related papers  (2020-10-01T16:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.