AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images

• URL: http://arxiv.org/abs/2606.31077v2
• Date: Wed, 01 Jul 2026 08:40:34 GMT
• Title: AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images
• Abstract summary: We propose AnyMatch to generate rich multi-modal training data at minimal cost.<n>AnyMatch integrates monocular depth estimation, 3D reprojection, diffusion-based inpainting, and crossmodal image translation.<n>We construct Any-syn, a large-scale synthetic multi-modal dataset using AnyMatch.
• Score: 46.95222327089868
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-modal image matching is essential for visual localization and multi-sensor fusion, but it is hindered by the scarcity of large-scale training data with precise geometric annotations. Existing real-world datasets suffer from prohibitive costs, limited scene diversity, and errors in SfM-MVS pipelines, while synthetic methods struggle to maintain 3D geometric consistency or achieve photorealistic appearance. To address this, we propose AnyMatch, a novel framework that leverages abundant, easily accessible single-view images at minimal cost to generate rich multi-modal training data. AnyMatch integrates monocular depth estimation, 3D reprojection, diffusion-based inpainting, and crossmodal image translation to synthesize multi-view, multi-modal image pairs with 3D geometric fidelity. Crucially, our method provides annotations that strictly adhere to 3D geometric consistency through explicit 3D reprojection, avoiding SfM-MVS error accumulation. Furthermore, AnyMatch offers strong scalability, enabling controllable scene diversity and annotation difficulty via adjustable input and camera parameters. We construct Any-syn, a large-scale synthetic multi-modal dataset using AnyMatch. Experimental results show that matching networks (e.g., LoFTR, EDM, RoMa) fine-tuned on Any-syn achieve substantial performance gains on multi-modal benchmarks, exhibiting superior generalization and robustness compared to models trained on existing data.

Related papers

• UMI3D: Robust 3D Generation on Unconstrained Multi-Image Inputs via Simultaneous Focus Cross-Attention Routing [49.1757838265224]
  Recent 3D foundation models can generate high-quality assets from a single image, but degrade markedly on unconstrained multi-image inputs.<n>We propose UMI3D, a training-free and plug-and-play framework that restructures cross-attention for unconstrained multi-image 3D generation.
  arXiv  Detail & Related papers  (2026-07-27T11:44:05Z)
• FusionBERT: Multi-View Image-3D Retrieval via Cross-Attention Visual Fusion and Normal-Aware 3D Encoder [17.364270724125447]
  FusionBERT is a novel framework for image-3D multimodal retrieval.<n>It uses a cross-attention-based multi-view visual aggregator to adaptively integrate features from multi-view images of an object.<n>It achieves significantly higher retrieval accuracy than SOTA multimodal large models under both single-view and multi-view settings.
  arXiv  Detail & Related papers  (2026-04-02T23:36:33Z)
• MuSASplat: Efficient Sparse-View 3D Gaussian Splats via Lightweight Multi-Scale Adaptation [92.57609195819647]
  MuSASplat is a novel framework that dramatically reduces the computational burden of training pose-free feed-forward 3D Gaussian splats models.<n>Central to our approach is a lightweight Multi-Scale Adapter that enables efficient fine-tuning of ViT-based architectures with only a small fraction of training parameters.
  arXiv  Detail & Related papers  (2025-12-08T04:56:46Z)
• MSMA: Multi-Scale Feature Fusion For Multi-Attribute 3D Face Reconstruction From Unconstrained Images [0.0]
  Reconstructing 3D face from a single unconstrained image remains a challenging problem due to diverse conditions in unconstrained environments.<n>We propose a Multi-Scale Feature Fusion with Multi-Attribute framework for 3D face reconstruction from unconstrained images.
  arXiv  Detail & Related papers  (2025-09-15T10:30:08Z)
• GenS: Generalizable Neural Surface Reconstruction from Multi-View Images [20.184657468900852]
  GenS is an end-to-end generalizable neural surface reconstruction model. Our representation is more powerful, which can recover high-frequency details while maintaining global smoothness. Experiments on popular benchmarks show that our model can generalize well to new scenes.
  arXiv  Detail & Related papers  (2024-06-04T17:13:10Z)
• IT3D: Improved Text-to-3D Generation with Explicit View Synthesis [71.68595192524843]
  This study presents a novel strategy that leverages explicitly synthesized multi-view images to address these issues. Our approach involves the utilization of image-to-image pipelines, empowered by LDMs, to generate posed high-quality images. For the incorporated discriminator, the synthesized multi-view images are considered real data, while the renderings of the optimized 3D models function as fake data.
  arXiv  Detail & Related papers  (2023-08-22T14:39:17Z)
• Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation [72.94143731623117]
  Existing methods simply align 3D representations with single-view 2D images and coarse-grained parent category text. Insufficient synergy neglects the idea that a robust 3D representation should align with the joint vision-language space. We propose a multi-view joint modality modeling approach, termed JM3D, to obtain a unified representation for point cloud, text, and image.
  arXiv  Detail & Related papers  (2023-08-06T01:11:40Z)
• Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis [48.33860286920389]
  3D-aware image synthesis aims to generate images of objects from multiple views by learning a 3D representation. Existing approaches lack geometry constraints, hence usually fail to generate multi-view consistent images. We propose Multi-View Consistent Generative Adrial Networks (MVCGAN) for high-quality 3D-aware image synthesis with geometry constraints.
  arXiv  Detail & Related papers  (2022-04-13T11:23:09Z)
• DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras [63.186486240525554]
  DeepMultiCap is a novel method for multi-person performance capture using sparse multi-view cameras. Our method can capture time varying surface details without the need of using pre-scanned template models.
  arXiv  Detail & Related papers  (2021-05-01T14:32:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.