Think Bright, Diffuse Nice: Enhancing T2I-ICL via Inductive-Bias Hint Instruction and Query Contrastive Decoding

• URL: http://arxiv.org/abs/2601.06169v1
• Date: Wed, 07 Jan 2026 06:39:45 GMT
• Title: Think Bright, Diffuse Nice: Enhancing T2I-ICL via Inductive-Bias Hint Instruction and Query Contrastive Decoding
• Abstract summary: Text-to-Image In-Context Learning enables customized image synthesis via interleaved text-image examples.<n>Existing methods rely on tailored training, which limits flexibility and raises deployment costs.<n>We propose TBDN, a training-free framework integrating two complementary closed-loop mechanisms.
• Score: 10.961445998450008
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-Image In-Context Learning (T2I-ICL) enables customized image synthesis via interleaved text-image examples but faces two mutually reinforcing bottlenecks, compliance failure and prior-dominated hallucination, that form a vicious cycle degrading generation quality. Existing methods rely on tailored training, which limits flexibility and raises deployment costs. To address these challenges effectively, we propose TBDN, a training-free framework integrating two complementary closed-loop mechanisms: Hint Instruction (HI) and Query Contrastive Decoding (QCD). HI injects task-aware inductive bias via lightweight prompt engineering to anchor models on contextual mapping rules, thereby mitigating compliance failure. QCD adjusts the decoding distributions of language models by contrasting full-input and query-omitted distributions, suppressing prior-dominated hallucination. TBDN achieves State-of-the-Art performance on CoBSAT and Text-to-Image Fast Mini-ImageNet, with robust generalization across model backbones, prompt designs, and hyperparameters. It also maintains promising performance in concept preservation and prompt following on Dreambench++. By breaking the two bottlenecks, TBDN establishes a simple yet effective framework for efficient and reliable T2I-ICL.

Related papers

• T2LDM++: A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation [63.89237657631886]
  Self-Conditioned Representation Guidance (SCRG) is used to alleviate object over-smoothing.<n>We construct two high-quality Text-LiDAR benchmarks using a generalized strategy of geometric annotations.<n>T2LDM++ can generate realistic LiDAR scenes with rich geometric details in unconditional and conditional settings.
  arXiv  Detail & Related papers  (2026-06-29T11:23:11Z)
• LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck [28.83441191878705]
  We propose LaME, which formulates embedding-oriented latent reasoning as a weakly supervised information bottleneck.<n>LaME employs K learnable reason tokens as a fixed-capacity bottleneck, completing all reasoning within a single forward pass.<n> Experiments on MMEB-v2 and MRMR show that LaME achieves competitive performance, surpassing some explicit CoT-based models.
  arXiv  Detail & Related papers  (2026-06-11T08:47:25Z)
• DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models [78.62380562116135]
  We propose DynT2I-Eval, a fully automated dynamic evaluation framework for text-to-image (T2I) models.<n>It constructs a structured visual semantic space from long-form descriptions, decomposing prompts into controllable dimensions.<n>DynT2I-Eval evaluates model performance across text alignment, perceptual quality, and aesthetics.
  arXiv  Detail & Related papers  (2026-05-07T12:53:51Z)
• Instruction-Evidence Contrastive Dual-Stream Decoding for Grounded Vision-Language Reasoning [3.944710336786154]
  Vision-Language Models (VLMs) exhibit strong performance in instruction following and open-ended vision-language reasoning.<n>They frequently generate fluent outputs that are weakly grounded in visual evidence.<n>We propose a decoding framework that balances linguistic informativeness and visual faithfulness during generation.
  arXiv  Detail & Related papers  (2026-04-28T16:18:31Z)
• MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding [32.0980373186604]
  We argue that left-to-right causal generation is an artifact of serialization rather than an intrinsic property of the task.<n>We propose MinerU-Diffusion, a unified diffusion-based framework that replaces autoregressive sequential decoding with parallel diffusion denoising.<n>Experiments demonstrate that MinerU-Diffusion consistently improves robustness while achieving up to 3.2x faster decoding compared to autoregressive baselines.
  arXiv  Detail & Related papers  (2026-03-23T18:25:17Z)
• End-to-End Training for Unified Tokenization and Latent Denoising [82.91537591286554]
  We propose UNITE - an autoencoder architecture for unified tokenization and latent diffusion.<n>UNITE consists of a Generative that serves as both image tokenizer and latent generator via weight sharing.<n>We show that single stage joint training of tokenization & generation from scratch is feasible.
  arXiv  Detail & Related papers  (2026-03-23T17:59:49Z)
• Training-free Context-adaptive Attention for Efficient Long Context Modeling [57.703159205740185]
  Training-free Context-adaptive Attention (TCA-Attention) is a training-free sparse attention mechanism that selectively attends to only the informative tokens for efficient long-context inference.<n>TCA-Attention achieves a 2.8$times$ speedup and reduces KV cache by 61% at 128K context length while maintaining performance comparable to full attention.
  arXiv  Detail & Related papers  (2025-12-10T01:54:57Z)
• RISE: Enhancing VLM Image Annotation with Self-Supervised Reasoning [10.797460135169763]
  Supervised Fine-Tuning (SFT) focuses solely on annotation outcomes, ignoring underlying rationales.<n>Visual Reinforcement Fine-Tuning (Visual-RFT) produces inconsistent Chains of Thought (CoTs) due to the absence of high-quality, verified CoTs during pre-training.<n>We introduce RISE (Reason-Inspire-Strengthen-Expertise), a two-stage framework to overcome these limitations.
  arXiv  Detail & Related papers  (2025-08-17T17:24:35Z)
• LumiGen: An LVLM-Enhanced Iterative Framework for Fine-Grained Text-to-Image Generation [1.124958340749622]
  Vision-Language Models (LVLMs) have demonstrated powerful capabilities in cross-modal understanding and instruction following.<n>LumiGen is a novel LVLM-enhanced iterative framework designed to elevate T2I model performance.<n>LumiGen achieves a superior average score of 3.08, outperforming state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-08-05T20:53:43Z)
• Beyond Degradation Redundancy: Contrastive Prompt Learning for All-in-One Image Restoration [109.38288333994407]
  Contrastive Prompt Learning (CPL) is a novel framework that fundamentally enhances prompt-task alignment.<n>Our framework establishes new state-of-the-art performance while maintaining parameter efficiency, offering a principled solution for unified image restoration.
  arXiv  Detail & Related papers  (2025-04-14T08:24:57Z)
• Training-Free Layout-to-Image Generation with Marginal Attention Constraints [73.55660250459132]
  We propose a training-free layout-to-image (L2I) approach, which eliminates the need for additional modules or fine-tuning.<n>Specifically, we use text-visual cross-attention feature maps to quantify inconsistencies between the layout of the generated images and the provided instructions.<n>We leverage pixel-to-pixel correlations in the self-attention feature maps to align cross-attention maps and combine three loss functions constrained by boundary attention to update latent features.
  arXiv  Detail & Related papers  (2024-11-15T05:44:45Z)
• Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding [14.175444025026508]
  Large language models (LLMs) have demonstrated remarkable capabilities in tasks requiring chain-of-thought (CoT) prompting. generating the full CoT process results in significantly longer output sequences, leading to increased computational costs and latency during inference. We propose a novel approach to compress the CoT process through semantic alignment, enabling more efficient decoding while preserving the benefits of CoT reasoning.
  arXiv  Detail & Related papers  (2024-09-13T06:29:20Z)
• Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Models [59.16287352266203]
  We introduce Adaptive Prompt-Tailored Pruning (APTP), a novel prompt-based pruning method for text-to-image (T2I) models.<n>APTP learns to determine the required capacity for an input text prompt and routes it to an architecture code, given a total desired compute budget for prompts.<n>APTP outperforms the single-model pruning baselines in terms of FID, CLIP, and CMMD scores.
  arXiv  Detail & Related papers  (2024-06-17T19:22:04Z)
• DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior [70.46245698746874]
  We present DiffBIR, a general restoration pipeline that could handle different blind image restoration tasks. DiffBIR decouples blind image restoration problem into two stages: 1) degradation removal: removing image-independent content; 2) information regeneration: generating the lost image content. In the first stage, we use restoration modules to remove degradations and obtain high-fidelity restored results. For the second stage, we propose IRControlNet that leverages the generative ability of latent diffusion models to generate realistic details.
  arXiv  Detail & Related papers  (2023-08-29T07:11:52Z)
• Noisy-Correspondence Learning for Text-to-Image Person Re-identification [50.07634676709067]
  We propose a novel Robust Dual Embedding method (RDE) to learn robust visual-semantic associations even with noisy correspondences. Our method achieves state-of-the-art results both with and without synthetic noisy correspondences on three datasets.
  arXiv  Detail & Related papers  (2023-08-19T05:34:13Z)
• Consistency-guided Prompt Learning for Vision-Language Models [23.4909421082857]
  We propose Consistency-guided Prompt learning (CoPrompt), a new fine-tuning method for vision-language models. Our approach improves the generalization of large foundation models when fine-tuned on downstream tasks in a few-shot setting.
  arXiv  Detail & Related papers  (2023-06-01T23:20:47Z)
• TIME: Text and Image Mutual-Translation Adversarial Networks [55.1298552773457]
  We propose Text and Image Mutual-Translation Adversarial Networks (TIME) TIME learns a T2I generator G and an image captioning discriminator D under the Generative Adversarial Network framework. In experiments, TIME achieves state-of-the-art (SOTA) performance on the CUB and MS-COCO dataset.
  arXiv  Detail & Related papers  (2020-05-27T06:40:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.