Deploy DINO with Many-to-Many Association

• URL: http://arxiv.org/abs/2604.23670v1
• Date: Sun, 26 Apr 2026 12:12:25 GMT
• Title: Deploy DINO with Many-to-Many Association
• Abstract summary: We introduce a novel likelihood perspective, which interprets the existing method as a zeroth-order approximation of otherwise intractable likelihood calculation.<n>We propose a faster and finer-grained robust mechanism, termed as Harmonic Consensus Maximization (HCM)
• Score: 7.8674424739322175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Motivated by the limited generalization of supervised image matching models to unseen image domains, we explore the zero-shot deployment of DINO features for this task. The generalist visual representation extracted from DINO has inherent ambiguity when used to match feature points among semantically similar instances, prompting us to adopt a many-to-many (m-to-m) matching paradigm. However, the existing robust mechanism under m-to-m data association is computationally heavy, which requires finding a maximum-cardinality matching in the inlier association graph for each parameter evaluation. To address this inefficiency, we introduce a novel likelihood perspective, which interprets the existing method as a zeroth-order approximation of otherwise intractable likelihood calculation,and inspires us to propose a faster and finer-grained robust mechanism, termed as Harmonic Consensus Maximization (HCM). Take camera pose estimation as an exemplifying downstream task, we demonstrate that general-purpose visual features, used out of the box without any adaptation, can compete with specialized matching models on out-of-distribution datasets when mated with m-to-m association and the HCM mechanism.

Related papers

• Collaborative Memory Augmentation for Generative Recommendation [49.76630658239287]
  We propose textbfOMEGA, a cOllaborative MEmory augmentation framework for Generative recommendAtion.<n>We first introduce a latent context compression method that utilizes learnable query tokens to distill sequential user behavior into compact representations.<n>These compressed representations are aggregated into a collaborative memory bank, serving as an explicit repository of global behavioral patterns.
  arXiv  Detail & Related papers  (2026-08-02T15:34:43Z)
• Where Does Generative Difficulty Reside? An Empirical Study of Target Representations [62.54876287800644]
  The target representation defines the distribution an image generator must learn, yet it is often treated as an interchangeable interface.<n>We study raw pixels, SD-VAE latents and DINOv2 as well as MAE representation-autoencoder features within a unified masked autoregressive rectified-flow model.<n>Our results show that compression, reconstruction fidelity, token dimensionality, and visible semantic clustering do not individually predict generative behavior.
  arXiv  Detail & Related papers  (2026-08-01T12:32:31Z)
• Model Merging as Probabilistic Inference in Fine-Tuning Parameter Space [16.602503149462738]
  We formulate model merging from a new perspective of probabilistic inference under a product-of-experts (PoE) scenario.<n>We show that several existing model merging methods arise as special cases of our framework under energy designs.<n>We address this with a heavy-tailed PoE design based on Cauchy experts, which better captures the observed residual behavior while admitting a provably convergent inference procedure.
  arXiv  Detail & Related papers  (2026-07-02T04:30:51Z)
• Maximum Likelihood Reconstruction for Multi-Look Digital Holography with Markov-Modeled Speckle Correlation [18.93229414605695]
  Multi-look acquisition is a widely used strategy for reducing speckle noise in coherent imaging systems such as digital holography.<n>In practice, hardware constraints limit measurement diversity, leading to inter-look correlation that degrades the performance of conventional methods.<n>We study the reconstruction of speckle-free reflectivity from complex-valued multi-look measurements in the presence of correlated speckle.
  arXiv  Detail & Related papers  (2026-04-22T03:36:52Z)
• One CT Unified Model Training Framework to Rule All Scanning Protocols [32.68329101435685]
  Non-ideal measurement computed tomography (NICT) is expanding the clinical use of CT.<n>Most methods require paired data, which is an impractical demand due to inevitable organ motion.<n>We propose an Uncertainty-Guided Manifold Smoothing (UMS) framework to bridge the gaps between sub-manifolds.
  arXiv  Detail & Related papers  (2026-03-16T09:26:32Z)
• Mind the Gap: Aligning Vision Foundation Models to Image Feature Matching [31.42132290162457]
  We introduce a new framework called IMD (Image feature Matching with a pre-trained Diffusion model) with two parts.<n>Unlike the dominant solutions employing contrastive-learning based foundation models that emphasize global semantics, we integrate the generative-based diffusion models.<n>Our proposed IMD establishes a new state-of-the-art in commonly evaluated benchmarks, and the superior 12% improvement in IMIM indicates our method efficiently mitigates the misalignment.
  arXiv  Detail & Related papers  (2025-07-14T14:28:15Z)
• PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Models [80.65273820998875]
  We present PanMatch, a versatile foundation model for robust correspondence matching.<n>Our key insight is that any two-frame correspondence matching task can be addressed within a 2D displacement estimation framework.<n>PanMatch achieves multi-task integration by endowing displacement estimation algorithms with unprecedented generalization capabilities.
  arXiv  Detail & Related papers  (2025-07-11T08:18:52Z)
• MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation [80.47072100963017]
  We introduce a novel and low-compute algorithm, Model Merging with Amortized Pareto Front (MAP)<n>MAP efficiently identifies a set of scaling coefficients for merging multiple models, reflecting the trade-offs involved.<n>We also introduce Bayesian MAP for scenarios with a relatively low number of tasks and Nested MAP for situations with a high number of tasks, further reducing the computational cost of evaluation.
  arXiv  Detail & Related papers  (2024-06-11T17:55:25Z)
• An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognition [49.45660055499103]
  Zero-shot human skeleton-based action recognition aims to construct a model that can recognize actions outside the categories seen during training. Previous research has focused on aligning sequences' visual and semantic spatial distributions. We introduce a new loss function sampling method to obtain a tight and robust representation.
  arXiv  Detail & Related papers  (2024-06-02T06:53:01Z)
• DEMO: A Statistical Perspective for Efficient Image-Text Matching [32.256725860652914]
  We introduce Distribution-based Structure Mining with Consistency Learning (DEMO) for efficient image-text matching. DEMO characterizes each image using multiple augmented views, which are considered as samples drawn from its intrinsic semantic distribution. In addition, we introduce collaborative consistency learning which not only preserves the similarity structure in the Hamming space but also encourages consistency between retrieval distribution from different directions.
  arXiv  Detail & Related papers  (2024-05-19T09:38:56Z)
• Representation Surgery for Multi-Task Model Merging [57.63643005215592]
  Multi-task learning (MTL) compresses the information from multiple tasks into a unified backbone to improve computational efficiency and generalization. Recent work directly merges multiple independently trained models to perform MTL instead of collecting their raw data for joint training. By visualizing the representation distribution of existing model merging schemes, we find that the merged model often suffers from the dilemma of representation bias.
  arXiv  Detail & Related papers  (2024-02-05T03:39:39Z)
• Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching [63.88319217738223]
  We present Matcher, a novel perception paradigm that utilizes off-the-shelf vision foundation models to address various perception tasks. Matcher demonstrates impressive generalization performance across various segmentation tasks, all without training. Our results further showcase the open-world generality and flexibility of Matcher when applied to images in the wild.
  arXiv  Detail & Related papers  (2023-05-22T17:59:43Z)
• Multiscale Deep Equilibrium Models [162.15362280927476]
  We propose a new class of implicit networks, the multiscale deep equilibrium model (MDEQ) An MDEQ directly solves for and backpropagates through the equilibrium points of multiple feature resolutions simultaneously. We illustrate the effectiveness of this approach on two large-scale vision tasks: ImageNet classification and semantic segmentation on high-resolution images from the Cityscapes dataset.
  arXiv  Detail & Related papers  (2020-06-15T18:07:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.