Why and How: Knowledge-Guided Learning for Cross-Spectral Image Patch Matching

• URL: http://arxiv.org/abs/2412.11161v1
• Date: Sun, 15 Dec 2024 11:59:23 GMT
• Title: Why and How: Knowledge-Guided Learning for Cross-Spectral Image Patch Matching
• Authors: Chuang Yu, Yunpeng Liu, Jinmiao Zhao, Xiangyu Yue,
• Abstract summary: Cross-spectral image patch matching based on feature relation learning has attracted extensive attention. We make the first attempt to explore a stable and efficient bridge between descriptor learning and metric learning. We construct a knowledge-guided learning network (KGL-Net) which achieves amazing performance improvements.
• Score: 7.699066648931588
• License:
• Abstract: Recently, cross-spectral image patch matching based on feature relation learning has attracted extensive attention. However, performance bottleneck problems have gradually emerged in existing methods. To address this challenge, we make the first attempt to explore a stable and efficient bridge between descriptor learning and metric learning, and construct a knowledge-guided learning network (KGL-Net), which achieves amazing performance improvements while abandoning complex network structures. Specifically, we find that there is feature extraction consistency between metric learning based on feature difference learning and descriptor learning based on Euclidean distance. This provides the foundation for bridge building. To ensure the stability and efficiency of the constructed bridge, on the one hand, we conduct an in-depth exploration of 20 combined network architectures. On the other hand, a feature-guided loss is constructed to achieve mutual guidance of features. In addition, unlike existing methods, we consider that the feature mapping ability of the metric branch should receive more attention. Therefore, a hard negative sample mining for metric learning (HNSM-M) strategy is constructed. To the best of our knowledge, this is the first time that hard negative sample mining for metric networks has been implemented and brings significant performance gains. Extensive experimental results show that our KGL-Net achieves SOTA performance in three different cross-spectral image patch matching scenarios. Our code are available at https://github.com/YuChuang1205/KGL-Net.

Related papers

• Class Anchor Margin Loss for Content-Based Image Retrieval [97.81742911657497]
  We propose a novel repeller-attractor loss that falls in the metric learning paradigm, yet directly optimize for the L2 metric without the need of generating pairs. We evaluate the proposed objective in the context of few-shot and full-set training on the CBIR task, by using both convolutional and transformer architectures.
  arXiv  Detail & Related papers  (2023-06-01T12:53:10Z)
• SuSana Distancia is all you need: Enforcing class separability in metric learning via two novel distance-based loss functions for few-shot image classification [0.9236074230806579]
  We propose two loss functions which consider the importance of the embedding vectors by looking at the intra-class and inter-class distance between the few data. Our results show a significant improvement in accuracy in the miniImagenNet benchmark compared to other metric-based few-shot learning methods by a margin of 2%.
  arXiv  Detail & Related papers  (2023-05-15T23:12:09Z)
• Learning to Learn with Indispensable Connections [6.040904021861969]
  We propose a novel meta-learning method called Meta-LTH that includes indispensible (necessary) connections. Our method improves the classification accuracy by approximately 2% (20-way 1-shot task setting) for omniglot dataset.
  arXiv  Detail & Related papers  (2023-04-06T04:53:13Z)
• Contrastive Learning of Features between Images and LiDAR [18.211513930388417]
  This work treats learning cross-modal features as a dense contrastive learning problem. To learn good features and not lose generality, we developed a variant of widely used PointNet++ architecture for images. We show that our models indeed learn information from both images as well as LiDAR by visualizing the features.
  arXiv  Detail & Related papers  (2022-06-24T04:35:23Z)
• Towards Interpretable Deep Metric Learning with Structural Matching [86.16700459215383]
  We present a deep interpretable metric learning (DIML) method for more transparent embedding learning. Our method is model-agnostic, which can be applied to off-the-shelf backbone networks and metric learning methods. We evaluate our method on three major benchmarks of deep metric learning including CUB200-2011, Cars196, and Stanford Online Products.
  arXiv  Detail & Related papers  (2021-08-12T17:59:09Z)
• Decoupled and Memory-Reinforced Networks: Towards Effective Feature Learning for One-Step Person Search [65.51181219410763]
  One-step methods have been developed to handle pedestrian detection and identification sub-tasks using a single network. There are two major challenges in the current one-step approaches. We propose a decoupled and memory-reinforced network (DMRNet) to overcome these problems.
  arXiv  Detail & Related papers  (2021-02-22T06:19:45Z)
• Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
  We introduce a fast optimization-based meta-learning method for few-shot classification. Our strategy enables important aspects of the base learner objective to be learned during meta-training. We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-10-01T15:59:31Z)
• MetricUNet: Synergistic Image- and Voxel-Level Learning for Precise CT Prostate Segmentation via Online Sampling [66.01558025094333]
  We propose a two-stage framework, with the first stage to quickly localize the prostate region and the second stage to precisely segment the prostate. We introduce a novel online metric learning module through voxel-wise sampling in the multi-task network. Our method can effectively learn more representative voxel-level features compared with the conventional learning methods with cross-entropy or Dice loss.
  arXiv  Detail & Related papers  (2020-05-15T10:37:02Z)
• ResNeSt: Split-Attention Networks [86.25490825631763]
  We present a modularized architecture, which applies the channel-wise attention on different network branches to leverage their success in capturing cross-feature interactions and learning diverse representations. Our model, named ResNeSt, outperforms EfficientNet in accuracy and latency trade-off on image classification.
  arXiv  Detail & Related papers  (2020-04-19T20:40:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.