DEIG: Detail-Enhanced Instance Generation with Fine-Grained Semantic Control

• URL: http://arxiv.org/abs/2602.18282v1
• Date: Fri, 20 Feb 2026 15:11:04 GMT
• Title: DEIG: Detail-Enhanced Instance Generation with Fine-Grained Semantic Control
• Abstract summary: DEIG is a novel framework for fine-grained and controllable multi-instance generation.<n>It generates visually coherent multi-instance scenes that precisely match rich, localized textual descriptions.<n>We introduce DEIG-Bench, a new benchmark with region-level annotations and multi-attribute prompts for both humans and objects.
• Score: 5.185018120081533
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-Instance Generation has advanced significantly in spatial placement and attribute binding. However, existing approaches still face challenges in fine-grained semantic understanding, particularly when dealing with complex textual descriptions. To overcome these limitations, we propose DEIG, a novel framework for fine-grained and controllable multi-instance generation. DEIG integrates an Instance Detail Extractor (IDE) that transforms text encoder embeddings into compact, instance-aware representations, and a Detail Fusion Module (DFM) that applies instance-based masked attention to prevent attribute leakage across instances. These components enable DEIG to generate visually coherent multi-instance scenes that precisely match rich, localized textual descriptions. To support fine-grained supervision, we construct a high-quality dataset with detailed, compositional instance captions generated by VLMs. We also introduce DEIG-Bench, a new benchmark with region-level annotations and multi-attribute prompts for both humans and objects. Experiments demonstrate that DEIG consistently outperforms existing approaches across multiple benchmarks in spatial consistency, semantic accuracy, and compositional generalization. Moreover, DEIG functions as a plug-and-play module, making it easily integrable into standard diffusion-based pipelines.

Related papers

• Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection [61.36976558603528]
  We propose a novel framework that generates modality-agnostic multi-modal prompts for the Segment Anything Model (SAM)<n>Specifically, we model multi-modal learning through interactions between a data-driven content domain and a knowledge-driven prompt domain.<n>We further introduce a lightweight Mask Refine Module to calibrate coarse predictions by incorporating fine-grained prompt cues.
  arXiv  Detail & Related papers  (2026-04-14T07:13:28Z)
• MIRAGE: Benchmarking and Aligning Multi-Instance Image Editing [9.297700134159916]
  We observe that state-of-the-art models suffer from severe over-editing and spatial misalignment when faced with multiple identical instances and composite instructions.<n>We propose Multi-Instance Regional Alignment via Guided Editing (MIRAGE), a training-free framework that enables precise, localized editing.<n>Our framework significantly outperforms existing methods in achieving precise instance-level modifications while preserving background consistency.
  arXiv  Detail & Related papers  (2026-04-06T21:20:00Z)
• GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection [54.19989440021701]
  Fine-grained open-vocabulary object detection (FG-OVD) aims to detect novel object categories described by attribute-rich texts.<n>FG-OVD achieves new state-of-the-art results, demonstrating the benefits of disentangled modeling and modular optimization.
  arXiv  Detail & Related papers  (2026-03-27T22:08:11Z)
• PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentation [58.1914505657064]
  This paper proposes a simple yet effective parallel cost aggregation (PCA-Seg) paradigm to alleviate the challenge of knowledge interference between class-level semantics and spatial context.<n>Experiments on eight benchmarks show that each parallel block in PCA-Seg adds merely 0.35M parameters while achieving state-of-the-art OSPS performance.
  arXiv  Detail & Related papers  (2026-03-18T09:26:43Z)
• AG-VAS: Anchor-Guided Zero-Shot Visual Anomaly Segmentation with Large Multimodal Models [21.682989096955467]
  AG-VAS (Anchor-Guided Visual Anomaly) is a new framework that expands the LMM vocabulary with three learnable semantic anchor tokens.<n>AG-VAS achieves consistent state-of-the-art performance in the zero-shot setting.
  arXiv  Detail & Related papers  (2026-03-01T22:25:23Z)
• InstanceV: Instance-Level Video Generation [97.01147000740417]
  We propose InstanceV, a video generation framework that enables i) instance-level control and ii) global semantic consistency.<n>We introduce the Shared Timestep-Adaptive Prompt Enhancement module, which connects local instances with global semantics.<n>Finally, we propose a new benchmark, named InstanceBench, which combines general video quality metrics with instance-aware metrics.
  arXiv  Detail & Related papers  (2025-11-28T12:52:37Z)
• RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering [50.42577862494645]
  We present RAG-IGBench, a benchmark designed to evaluate the task of Interleaved Generation based on Retrieval-Augmented Generation (RAG-IG) in open-domain question answering.<n>RAG-IG integrates multimodal large language models (MLLMs) with retrieval mechanisms, enabling the models to access external image-text information for generating coherent multimodal content.
  arXiv  Detail & Related papers  (2025-10-11T03:06:39Z)
• Improving Generalized Visual Grounding with Instance-aware Joint Learning [45.53531162436934]
  Generalized visual grounding tasks are designed to accommodate multi-target and non-target scenarios.<n>We propose InstanceVG, a framework equipped with instance-aware capabilities to tackle both GREC and GRES.<n>To instantiate the framework, we assign each instance query a prior reference point, which also serves as an additional basis for target matching.
  arXiv  Detail & Related papers  (2025-09-17T07:00:51Z)
• FOCUS: Unified Vision-Language Modeling for Interactive Editing Driven by Referential Segmentation [55.01077993490845]
  Recent Large Vision Language Models (LVLMs) demonstrate promising capabilities in unifying visual understanding and generative modeling.<n>We introduce FOCUS, a unified LVLM that integrates segmentation-aware perception and controllable object-centric generation within an end-to-end framework.
  arXiv  Detail & Related papers  (2025-06-20T07:46:40Z)
• NEXT: Multi-Grained Mixture of Experts via Text-Modulation for Multi-Modal Object Re-Identification [17.10113184019939]
  Multi-modal object Re-Identification (ReID) aims to obtain accurate identity features across heterogeneous modalities.<n>In this paper, we propose a reliable caption generation pipeline based on attribute confidence.<n>We also propose a novel ReID framework, named NEXT, to model diverse identity patterns.
  arXiv  Detail & Related papers  (2025-05-26T13:52:28Z)
• IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identification [60.38841251693781]
  We propose a novel framework to generate robust multi-modal object ReIDs.<n>Our framework uses Modal Prefixes and InverseNet to integrate multi-modal information with semantic guidance from inverted text.<n>Experiments on three multi-modal object ReID benchmarks demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2025-03-13T13:00:31Z)
• MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesis [33.52454028815209]
  We introduce the Multi-Instance Generation (MIG) task, which focuses on generating multiple instances within a single image.<n>MIG faces three main challenges: avoiding attribute leakage between instances, supporting diverse instance descriptions, and maintaining consistency in iterative generation.<n>We introduce the COCO-MIG and Multimodal-MIG benchmarks to evaluate these methods.
  arXiv  Detail & Related papers  (2024-07-02T14:59:37Z)
• MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis [22.27724733876081]
  We present a Multi-Instance Generation (MIG) task, simultaneously generating multiple instances with diverse controls in one image. We introduce an innovative approach named Multi-Instance Generation Controller (MIGC) to address the challenges of the MIG task. To evaluate how well generation models perform on the MIG task, we provide a COCO-MIG benchmark along with an evaluation pipeline.
  arXiv  Detail & Related papers  (2024-02-08T04:52:36Z)
• LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts [60.54912319612113]
  Diffusion-based generative models have significantly advanced text-to-image generation but encounter challenges when processing lengthy and intricate text prompts. We present a novel approach leveraging Large Language Models (LLMs) to extract critical components from text prompts. Our evaluation on complex prompts featuring multiple objects demonstrates a substantial improvement in recall compared to baseline diffusion models.
  arXiv  Detail & Related papers  (2023-10-16T17:57:37Z)
• M$^3$Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition [80.21796574234287]
  M$3$Net is a matching-based framework for few-shot fine-grained (FS-FG) action recognition. It incorporates textitmulti-view encoding, textitmulti-view matching, and textitmulti-view fusion to facilitate embedding encoding, similarity matching, and decision making. Explainable visualizations and experimental results demonstrate the superiority of M$3$Net in capturing fine-grained action details.
  arXiv  Detail & Related papers  (2023-08-06T09:15:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.