DAFM: Dynamic Adaptive Fusion for Multi-Model Collaboration in Composed Image Retrieval

• URL: http://arxiv.org/abs/2511.05020v1
• Date: Fri, 07 Nov 2025 06:51:10 GMT
• Title: DAFM: Dynamic Adaptive Fusion for Multi-Model Collaboration in Composed Image Retrieval
• Authors: Yawei Cai, Jiapeng Mi, Nan Ji, Haotian Rong, Yawei Zhang, Zhangti Li, Wenbin Guo, Rensong Xie,
• Abstract summary: We propose Dynamic Adaptive Fusion (DAFM) for multi-model collaboration in Composed Image Retrieval (CIR)<n>Our method achieves a Recall@10 of 93.21 and an Rmean of 84.43 on CIRR, and an average Rmean of 67.48 on FashionIQ, surpassing recent strong baselines by up to 4.5%.
• Score: 2.330678113289435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Composed Image Retrieval (CIR) is a cross-modal task that aims to retrieve target images from large-scale databases using a reference image and a modification text. Most existing methods rely on a single model to perform feature fusion and similarity matching. However, this paradigm faces two major challenges. First, one model alone can't see the whole picture and the tiny details at the same time; it has to handle different tasks with the same weights, so it often misses the small but important links between image and text. Second, the absence of dynamic weight allocation prevents adaptive leveraging of complementary model strengths, so the resulting embedding drifts away from the target and misleads the nearest-neighbor search in CIR. To address these limitations, we propose Dynamic Adaptive Fusion (DAFM) for multi-model collaboration in CIR. Rather than optimizing a single method in isolation, DAFM exploits the complementary strengths of heterogeneous models and adaptively rebalances their contributions. This not only maximizes retrieval accuracy but also ensures that the performance gains are independent of the fusion order, highlighting the robustness of our approach. Experiments on the CIRR and FashionIQ benchmarks demonstrate consistent improvements. Our method achieves a Recall@10 of 93.21 and an Rmean of 84.43 on CIRR, and an average Rmean of 67.48 on FashionIQ, surpassing recent strong baselines by up to 4.5%. These results confirm that dynamic multi-model collaboration provides an effective and general solution for CIR.

Related papers

• Plug, Play, and Fortify: A Low-Cost Module for Robust Multimodal Image Understanding Models [6.350443894942629]
  Multimodal Weight Allocation Module (MWAM) is a plug-and-play component that dynamically re-balances the contribution of each branch during training.<n>MWAM delivers consistent performance gains across a wide range of tasks and modality combinations.
  arXiv  Detail & Related papers  (2026-02-26T05:51:41Z)
• FusionFM: All-in-One Multi-Modal Image Fusion with Flow Matching [42.22268167379098]
  We formulate image fusion as a direct probabilistic transport from source modalities to the fused image distribution.<n>We employ a task-aware selection function to select the most reliable pseudo-labels for each task.<n>For multi-task scenarios, we integrate elastic weight consolidation and experience replay mechanisms to preserve cross-task performance.
  arXiv  Detail & Related papers  (2025-11-17T02:56:48Z)
• Towards Reversible Model Merging For Low-rank Weights [5.100622189286672]
  Model merging aims to combine multiple fine-tuned models into a single set of weights that performs well across all source tasks.<n>We show that applying conventional merging methods to low-rank weights leads to severe performance degradation in the merged model.<n>We propose a fundamentally different approach: instead of collapsing all adapters into one set of weights, we construct a compact basis.<n>This reframes merging as generating a reconstruction-capable model space rather than producing a single merged model.
  arXiv  Detail & Related papers  (2025-10-15T23:22:38Z)
• Mind the Gap: Aligning Vision Foundation Models to Image Feature Matching [31.42132290162457]
  We introduce a new framework called IMD (Image feature Matching with a pre-trained Diffusion model) with two parts.<n>Unlike the dominant solutions employing contrastive-learning based foundation models that emphasize global semantics, we integrate the generative-based diffusion models.<n>Our proposed IMD establishes a new state-of-the-art in commonly evaluated benchmarks, and the superior 12% improvement in IMIM indicates our method efficiently mitigates the misalignment.
  arXiv  Detail & Related papers  (2025-07-14T14:28:15Z)
• Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution [88.20464308588889]
  We propose a Structural Similarity-Inspired Unfolding (SSIU) method for efficient image SR.<n>This method is designed through unfolding an SR optimization function constrained by structural similarity.<n>Our model outperforms current state-of-the-art models, boasting lower parameter counts and reduced memory consumption.
  arXiv  Detail & Related papers  (2025-06-13T14:29:40Z)
• PS-ReID: Advancing Person Re-Identification and Precise Segmentation with Multimodal Retrieval [38.530536338075684]
  Person re-identification (ReID) plays a critical role in applications such as security surveillance and criminal investigations.<n>We propose bf PS-ReID, a multimodal model that combines image and text inputs to enhance ReID performance.<n> Experimental results demonstrate that PS-ReID significantly outperforms unimodal query-based models in both ReID and segmentation tasks.
  arXiv  Detail & Related papers  (2025-03-27T15:14:03Z)
• CoLLM: A Large Language Model for Composed Image Retrieval [76.29725148964368]
  Composed Image Retrieval (CIR) is a complex task that aims to retrieve images based on a multimodal query.<n>We present CoLLM, a one-stop framework that generates triplets on-the-fly from image-caption pairs.<n>We leverage Large Language Models (LLMs) to generate joint embeddings of reference images and modification texts.
  arXiv  Detail & Related papers  (2025-03-25T17:59:50Z)
• CTSR: Controllable Fidelity-Realness Trade-off Distillation for Real-World Image Super Resolution [52.93785843453579]
  Real-world image super-resolution is a critical image processing task, where two key evaluation criteria are the fidelity to the original image and the visual realness of the generated results.<n>We propose a distillation-based approach that leverages the geometric decomposition of both fidelity and realness, alongside the performance advantages of multiple teacher models.<n> Experiments conducted on several real-world image super-resolution benchmarks demonstrate that our method surpasses existing state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-03-18T14:06:39Z)
• Weight Scope Alignment: A Frustratingly Easy Method for Model Merging [40.080926444789085]
  Non-I.I.D. data poses a huge challenge for averaging-based model fusion. In this paper, we reveal variations in weight scope under different training conditions, shedding light on its influence on model merging. Fortunately, the parameters in each layer basically follow the Gaussian distribution, which inspires a novel and simple regularization approach.
  arXiv  Detail & Related papers  (2024-08-22T09:13:27Z)
• EMR-Merging: Tuning-Free High-Performance Model Merging [55.03509900949149]
  We show that Elect, Mask & Rescale-Merging (EMR-Merging) shows outstanding performance compared to existing merging methods. EMR-Merging is tuning-free, thus requiring no data availability or any additional training while showing impressive performance.
  arXiv  Detail & Related papers  (2024-05-23T05:25:45Z)
• Deep Equilibrium Diffusion Restoration with Parallel Sampling [120.15039525209106]
  Diffusion model-based image restoration (IR) aims to use diffusion models to recover high-quality (HQ) images from degraded images, achieving promising performance. Most existing methods need long serial sampling chains to restore HQ images step-by-step, resulting in expensive sampling time and high computation costs. In this work, we aim to rethink the diffusion model-based IR models through a different perspective, i.e., a deep equilibrium (DEQ) fixed point system, called DeqIR.
  arXiv  Detail & Related papers  (2023-11-20T08:27:56Z)
• Hierarchical Integration Diffusion Model for Realistic Image Deblurring [71.76410266003917]
  Diffusion models (DMs) have been introduced in image deblurring and exhibited promising performance. We propose the Hierarchical Integration Diffusion Model (HI-Diff), for realistic image deblurring. Experiments on synthetic and real-world blur datasets demonstrate that our HI-Diff outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-22T12:18:20Z)
• VMLoc: Variational Fusion For Learning-Based Multimodal Camera Localization [36.11674066519118]
  We propose an end-to-end framework, termed VMLoc, to fuse different sensor inputs into a common latent space.<n>Unlike previous multimodal variational works directly adapting the objective function of vanilla variational auto-encoder, we show how camera localization can be accurately estimated.
  arXiv  Detail & Related papers  (2020-03-12T14:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.