ADAPT: Attention Driven Adaptive Prompt Scheduling and InTerpolating Orthogonal Complements for Rare Concepts Generation

• URL: http://arxiv.org/abs/2603.19157v1
• Date: Thu, 19 Mar 2026 17:11:49 GMT
• Title: ADAPT: Attention Driven Adaptive Prompt Scheduling and InTerpolating Orthogonal Complements for Rare Concepts Generation
• Abstract summary: ADAPT is a training-free framework that deterministically plans and semantically aligns prompt schedules.<n>We demonstrate that ADAPT achieves superior performance in RareBench and accurately reflects the semantic information of rare attributes.
• Score: 7.233066974580282
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generating rare compositional concepts in text-to-image synthesis remains a challenge for diffusion models, particularly for attributes that are uncommon in the training data. While recent approaches, such as R2F, address this challenge by utilizing LLM for prompt scheduling, they suffer from inherent variance due to the randomness of language models and suboptimal guidance from iterative text embedding switching. To address these problems, we propose the ADAPT framework, a training-free framework that deterministically plans and semantically aligns prompt schedules, providing consistent guidance to enhance the composition of rare concepts. By leveraging attention scores and orthogonal components, ADAPT significantly enhances compositional generation of rare concepts in the RareBench benchmark without additional training or fine-tuning. Through comprehensive experiments, we demonstrate that ADAPT achieves superior performance in RareBench and accurately reflects the semantic information of rare attributes, providing deterministic and precise control over the generation of rare compositions without compromising visual integrity.

Related papers

• Joint-Embedding Predictive Architecture for Sensor-based Activity Recognition [3.775283002059578]
  This paper proposes a Joint Embedding Predictive Architecture framework tailored for sensor-based human activity recognition (HAR)<n>The proposed framework features an encoder designed to explicitly model both the fine-grained local temporal representations within individual window and the long-term temporal sequence of adjacent windows.<n>The results show that high-quality representations are successfully learned by the proposed framework.
  arXiv  Detail & Related papers  (2026-07-17T05:28:22Z)
• Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding [56.38477453373368]
  LazyMCoT is a dynamic and training-free framework that adaptively allocates visual grounding efforts based on sample difficulty.<n>LazyMCoT rivals training-based approaches by simultaneously improving reasoning accuracy and reducing average inference latency.
  arXiv  Detail & Related papers  (2026-06-15T03:17:44Z)
• A Systematic Post-Train Framework for Video Generation [76.26555417456773]
  Large-scale video diffusion models have demonstrated impressive capabilities in generating high-resolution and semantically rich content.<n>A significant gap remains between their pretraining performance and real-world deployment requirements due to critical issues such as prompt sensitivity, temporal inconsistency, and prohibitive inference costs.<n>We propose a comprehensive post-training framework that systematically aligns pretrained models with user intentions through four synergistic stages.
  arXiv  Detail & Related papers  (2026-04-28T09:34:51Z)
• Adaptive Auxiliary Prompt Blending for Target-Faithful Diffusion Generation [6.826122099204317]
  We introduce Adaptive Auxiliary Prompt Blending (AAPB) to stabilize the diffusion process in low-density regions.<n>AAPB provides semantic support in rare concept generation and structural support in image editing.<n>We empirically show consistent improvements on the RareBench and FlowEdit datasets.
  arXiv  Detail & Related papers  (2026-03-19T17:12:03Z)
• Fine-tuning MLLMs Without Forgetting Is Easier Than You Think [72.59321247529975]
  We design a 2x2 experimental framework to assess model performance across in-distribution and out-of-distribution image and text inputs.<n>Our results show that appropriate regularization, such as constraining the number of trainable parameters or adopting a low learning rate, effectively prevents forgetting when dealing with out-of-distribution images.<n>We attribute this forgetting as task-specific overfitting and address this issue by introducing a data-hybrid training strategy.
  arXiv  Detail & Related papers  (2026-03-15T17:16:19Z)
• Unleashing the Power of Vision-Language Models for Long-Tailed Multi-Label Visual Recognition [55.189113121465816]
  We propose a novel correlation adaptation prompt network (CAPNET) for long-tailed multi-label visual recognition.<n>CAPNET explicitly models correlations from CLIP's textual encoder.<n>It improves generalization through test-time ensembling and realigns visual-textual modalities.
  arXiv  Detail & Related papers  (2025-11-25T18:57:28Z)
• Explaining Time Series Classifiers with PHAR: Rule Extraction and Fusion from Post-hoc Attributions [7.51289645756884]
  PHAR is a framework that transforms numeric feature attributions into structured, human-readable rules.<n>A dedicated rule fusion step consolidates rule sets using strategies like weighted selection and lasso-based refinement.<n>Experiments on UCR/UEA Time Series Classification Archive demonstrate that PHAR may improve interpretability, decision transparency, and practical applicability for TS classification tasks.
  arXiv  Detail & Related papers  (2025-08-03T09:45:40Z)
• Semantic-guided Fine-tuning of Foundation Model for Long-tailed Visual Recognition [38.74388860692423]
  We propose a novel approach, Semantic-guided fine-tuning of foundation model for long-tailed visual recognition (Sage)<n>We introduce an SG-Adapter that integrates class descriptions as semantic guidance to guide the fine-tuning of the visual encoder.<n>Experiments on benchmark datasets demonstrate the effectiveness of the proposed Sage in enhancing performance in long-tailed learning.
  arXiv  Detail & Related papers  (2025-07-17T05:47:19Z)
• StPR: Spatiotemporal Preservation and Routing for Exemplar-Free Video Class-Incremental Learning [79.44594332189018]
  Class-Incremental Learning (CIL) seeks to develop models that continuously learn new action categories over time without previously acquired knowledge.<n>Existing approaches either rely on forgetting, raising concerns over memory and privacy, or adapt static image-based methods that neglect temporal modeling.<n>We propose a unified and exemplar-free VCIL framework that explicitly disentangles and preserves information.
  arXiv  Detail & Related papers  (2025-05-20T06:46:51Z)
• PLOT-TAL: Prompt Learning with Optimal Transport for Few-Shot Temporal Action Localization [8.173421927978117]
  We introduce PLOT-TAL, a framework that finds globally optimal alignment between the prompt ensemble and the video's temporal features.<n>Our method establishes a new state-of-the-art on the challenging few-shot benchmarks of THUMOS'14 and EPIC-Kitchens, without requiring complex meta-learning.
  arXiv  Detail & Related papers  (2024-03-27T18:08:14Z)
• Self-regulating Prompts: Foundational Model Adaptation without Forgetting [112.66832145320434]
  We introduce a self-regularization framework for prompting called PromptSRC. PromptSRC guides the prompts to optimize for both task-specific and task-agnostic general representations.
  arXiv  Detail & Related papers  (2023-07-13T17:59:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.