Affinity guided Geometric Semi-Supervised Metric Learning

• URL: http://arxiv.org/abs/2002.12394v2
• Date: Fri, 6 Nov 2020 17:49:32 GMT
• Title: Affinity guided Geometric Semi-Supervised Metric Learning
• Authors: Ujjal Kr Dutta, Mehrtash Harandi and Chellu Chandra Sekhar
• Abstract summary: The motivation comes from the fact that apart from a few classical SSDML approaches learning a linear Mahalanobis metric, deep SSDML has not been studied. We first extend existing SSDML methods to their deep counterparts and then propose a new method to overcome their limitations. Our deep SSDML method with a novel affinity propagation based triplet mining strategy outperforms its competitors.
• Score: 27.955068939695042
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we revamp the forgotten classical Semi-Supervised Distance Metric Learning (SSDML) problem from a Riemannian geometric lens, to leverage stochastic optimization within a end-to-end deep framework. The motivation comes from the fact that apart from a few classical SSDML approaches learning a linear Mahalanobis metric, deep SSDML has not been studied. We first extend existing SSDML methods to their deep counterparts and then propose a new method to overcome their limitations. Due to the nature of constraints on our metric parameters, we leverage Riemannian optimization. Our deep SSDML method with a novel affinity propagation based triplet mining strategy outperforms its competitors.

Related papers

• Riemannian Geometric-based Meta Learning [8.365106891566725]
  "Learning to learn" aims to enable models to quickly adapt to new tasks with minimal data. Traditional methods like Model-Agnostic Meta-Learning (MAML) often struggle to capture complex learning dynamics. We propose Stiefel-MAML, which integrates Riemannian geometry by optimizing within the Stiefel manifold.
  arXiv  Detail & Related papers  (2025-03-14T01:34:55Z)
• Training Deep Learning Models with Norm-Constrained LMOs [56.00317694850397]
  We study optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball. We propose a new family of algorithms that uses the LMO to adapt to the geometry of the problem and, perhaps surprisingly, show that they can be applied to unconstrained problems.
  arXiv  Detail & Related papers  (2025-02-11T13:10:34Z)
• Towards Scalable Semantic Representation for Recommendation [65.06144407288127]
  Mixture-of-Codes is proposed to construct semantic IDs based on large language models (LLMs) Our method achieves superior discriminability and dimension robustness scalability, leading to the best scale-up performance in recommendations.
  arXiv  Detail & Related papers  (2024-10-12T15:10:56Z)
• Zeroth-Order Fine-Tuning of LLMs in Random Subspaces [66.27334633749734]
  As language models grow in size, memory demands for backpropagation increase. Zeroth-order (ZOZO) optimization methods offer a memory-efficient alternative. We show that SubZero enhances fine-tuning and achieves faster results compared to standard ZOZO approaches.
  arXiv  Detail & Related papers  (2024-10-11T17:01:43Z)
• RMLR: Extending Multinomial Logistic Regression into General Geometries [64.16104856124029]
  Our framework only requires minimal geometric properties, thus exhibiting broad applicability. We develop five families of SPD MLRs under five types of power-deformed metrics. On rotation matrices we propose Lie MLR based on the popular bi-invariant metric.
  arXiv  Detail & Related papers  (2024-09-28T18:38:21Z)
• FORML: A Riemannian Hessian-free Method for Meta-learning on Stiefel Manifolds [4.757859522106933]
  This paper introduces a Hessian-free approach that uses a first-order approximation of derivatives on the Stiefel manifold. Our method significantly reduces the computational load and memory footprint.
  arXiv  Detail & Related papers  (2024-02-28T10:57:30Z)
• Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark [166.40879020706151]
  This paper proposes a shift towards BP-free, zeroth-order (ZO) optimization as a solution for reducing memory costs during fine-tuning. Unlike traditional ZO-SGD methods, our work expands the exploration to a wider array of ZO optimization techniques. Our study unveils previously overlooked optimization principles, highlighting the importance of task alignment, the role of the forward gradient method, and the balance between algorithm complexity and fine-tuning performance.
  arXiv  Detail & Related papers  (2024-02-18T14:08:48Z)
• Riemannian Self-Attention Mechanism for SPD Networks [34.794770395408335]
  An SPD manifold self-attention mechanism (SMSA) is proposed in this paper. An SMSA-based geometric learning module (SMSA-GL) is designed for the sake of improving the discrimination of structured representations.
  arXiv  Detail & Related papers  (2023-11-28T12:34:46Z)
• Adaptive Log-Euclidean Metrics for SPD Matrix Learning [73.12655932115881]
  We propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM) The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks.
  arXiv  Detail & Related papers  (2023-03-26T18:31:52Z)
• SetMargin Loss applied to Deep Keystroke Biometrics with Circle Packing Interpretation [67.0845003374569]
  This work presents a new deep learning approach for keystroke biometrics based on a novel Distance Metric Learning method (DML) We prove experimentally the effectiveness of the proposed approach on a challenging task: keystroke biometric identification over a large set of 78,000 subjects.
  arXiv  Detail & Related papers  (2021-09-02T13:26:57Z)
• Semi-Supervised Metric Learning: A Deep Resurrection [22.918651280720855]
  Semi-Supervised DML (SSDML) tries to learn a metric using a few labeled examples, and abundantly available unlabeled examples. We propose a graph-based approach that first propagates the affinities between the pairs of examples. We impose Metricity constraint on the metric parameters, as it leads to a better performance.
  arXiv  Detail & Related papers  (2021-05-10T12:28:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.