Towards Self-Adaptive Metric Learning On the Fly

• URL: http://arxiv.org/abs/2104.01495v1
• Date: Sat, 3 Apr 2021 23:11:52 GMT
• Title: Towards Self-Adaptive Metric Learning On the Fly
• Authors: Yang Gao, Yi-Fan Li, Swarup Chandra, Latifur Khan, Bhavani Thuraisingham
• Abstract summary: We aim to address the open challenge of "Online Adaptive Metric Learning" (OAML) for learning adaptive metric functions on the fly. Unlike traditional online metric learning methods, OAML is significantly more challenging since the learned metric could be non-linear and the model has to be self-adaptive. We present a new online metric learning framework that attempts to tackle the challenge by learning an ANN-based metric with adaptive model complexity from a stream of constraints.
• Score: 16.61982837441342
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Good quality similarity metrics can significantly facilitate the performance of many large-scale, real-world applications. Existing studies have proposed various solutions to learn a Mahalanobis or bilinear metric in an online fashion by either restricting distances between similar (dissimilar) pairs to be smaller (larger) than a given lower (upper) bound or requiring similar instances to be separated from dissimilar instances with a given margin. However, these linear metrics learned by leveraging fixed bounds or margins may not perform well in real-world applications, especially when data distributions are complex. We aim to address the open challenge of "Online Adaptive Metric Learning" (OAML) for learning adaptive metric functions on the fly. Unlike traditional online metric learning methods, OAML is significantly more challenging since the learned metric could be non-linear and the model has to be self-adaptive as more instances are observed. In this paper, we present a new online metric learning framework that attempts to tackle the challenge by learning an ANN-based metric with adaptive model complexity from a stream of constraints. In particular, we propose a novel Adaptive-Bound Triplet Loss (ABTL) to effectively utilize the input constraints and present a novel Adaptive Hedge Update (AHU) method for online updating the model parameters. We empirically validate the effectiveness and efficacy of our framework on various applications such as real-world image classification, facial verification, and image retrieval.

Related papers

• Unlearning with Control: Assessing Real-world Utility for Large Language Model Unlearning [97.2995389188179]
  Recent research has begun to approach large language models (LLMs) unlearning via gradient ascent (GA) Despite their simplicity and efficiency, we suggest that GA-based methods face the propensity towards excessive unlearning. We propose several controlling methods that can regulate the extent of excessive unlearning.
  arXiv  Detail & Related papers  (2024-06-13T14:41:00Z)
• AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy [27.443439653087662]
  This paper introduces AQUILA (adaptive quantization in device selection strategy), a novel adaptive framework devised to handle these issues. AQUILA integrates a sophisticated device selection method that prioritizes the quality and usefulness of device updates. Our experiments demonstrate that AQUILA significantly decreases communication costs compared to existing methods.
  arXiv  Detail & Related papers  (2023-08-01T03:41:47Z)
• An Adaptive Plug-and-Play Network for Few-Shot Learning [12.023266104119289]
  Few-shot learning requires a model to classify new samples after learning from only a few samples. Deep networks and complex metrics tend to induce overfitting, making it difficult to further improve the performance. We propose plug-and-play model-adaptive resizer (MAR) and adaptive similarity metric (ASM) without any other losses.
  arXiv  Detail & Related papers  (2023-02-18T13:25:04Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• Adaptive Fairness-Aware Online Meta-Learning for Changing Environments [29.073555722548956]
  The fairness-aware online learning framework has arisen as a powerful tool for the continual lifelong learning setting. Existing methods make heavy use of the i.i.d assumption for data and hence provide static regret analysis for the framework. We propose a novel adaptive fairness-aware online meta-learning algorithm, namely FairSAOML, which is able to adapt to changing environments in both bias control and model precision.
  arXiv  Detail & Related papers  (2022-05-20T15:29:38Z)
• Adaptive Hierarchical Similarity Metric Learning with Noisy Labels [138.41576366096137]
  We propose an Adaptive Hierarchical Similarity Metric Learning method. It considers two noise-insensitive information, textiti.e., class-wise divergence and sample-wise consistency. Our method achieves state-of-the-art performance compared with current deep metric learning approaches.
  arXiv  Detail & Related papers  (2021-10-29T02:12:18Z)
• Low-Rank Robust Online Distance/Similarity Learning based on the Rescaled Hinge Loss [0.34376560669160383]
  Existing online methods usually assume training triplets or pairwise constraints are exist in advance. We formulate the online Distance-Similarity learning problem with the robust Rescaled hinge loss function. The proposed model is rather general and can be applied to any PA-based online Distance-Similarity algorithm.
  arXiv  Detail & Related papers  (2020-10-07T08:38:34Z)
• Optimization-driven Machine Learning for Intelligent Reflecting Surfaces Assisted Wireless Networks [82.33619654835348]
  Intelligent surface (IRS) has been employed to reshape the wireless channels by controlling individual scattering elements' phase shifts. Due to the large size of scattering elements, the passive beamforming is typically challenged by the high computational complexity. In this article, we focus on machine learning (ML) approaches for performance in IRS-assisted wireless networks.
  arXiv  Detail & Related papers  (2020-08-29T08:39:43Z)
• ECML: An Ensemble Cascade Metric Learning Mechanism towards Face Verification [50.137924223702264]
  In particular, hierarchical metric learning is executed in the cascade way to alleviate underfitting. Considering the feature distribution characteristics of faces, a robust Mahalanobis metric learning method (RMML) with closed-form solution is additionally proposed. EC-RMML is superior to state-of-the-art metric learning methods for face verification.
  arXiv  Detail & Related papers  (2020-07-11T08:47:07Z)
• A Multilayer Framework for Online Metric Learning [71.31889711244739]
  This paper proposes a multilayer framework for online metric learning to capture the nonlinear similarities among instances. A new Mahalanobis-based Online Metric Learning (MOML) algorithm is presented based on the passive-aggressive strategy and one-pass triplet construction strategy. The proposed MLOML enjoys several nice properties, indeed learns a metric progressively, and performs better on the benchmark datasets.
  arXiv  Detail & Related papers  (2018-05-15T01:10:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.