IDEAL: Independent Domain Embedding Augmentation Learning

• URL: http://arxiv.org/abs/2105.10112v1
• Date: Fri, 21 May 2021 03:40:24 GMT
• Title: IDEAL: Independent Domain Embedding Augmentation Learning
• Authors: Zhiyuan Chen, Guang Yao, Wennan Ma, Lin Xu
• Abstract summary: We develop a novel mechanism, the independent domain embedding augmentation learning (I) method. It can simultaneously learn multiple independent embedding spaces for transformations generated by multiple data domains. Our ISOP is to existing DML techniques and can be seamlessly combined with prior DML approaches for enhanced performance.
• Score: 8.376337907951012
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many efforts have been devoted to designing sampling, mining, and weighting strategies in high-level deep metric learning (DML) loss objectives. However, little attention has been paid to low-level but essential data transformation. In this paper, we develop a novel mechanism, the independent domain embedding augmentation learning ({IDEAL}) method. It can simultaneously learn multiple independent embedding spaces for multiple domains generated by predefined data transformations. Our IDEAL is orthogonal to existing DML techniques and can be seamlessly combined with prior DML approaches for enhanced performance. Empirical results on visual retrieval tasks demonstrate the superiority of the proposed method. For example, the IDEAL improves the performance of MS loss by a large margin, 84.5\% $\rightarrow$ 87.1\% on Cars-196, and 65.8\% $\rightarrow$ 69.5\% on CUB-200 at Recall$@1$. Our IDEAL with MS loss also achieves the new state-of-the-art performance on three image retrieval benchmarks, \ie, \emph{Cars-196}, \emph{CUB-200}, and \emph{SOP}. It outperforms the most recent DML approaches, such as Circle loss and XBM, significantly. The source code and pre-trained models of our method will be available at\emph{\url{https://github.com/emdata-ailab/IDEAL}}.

Related papers

• LESA: Learnable LLM Layer Scaling-Up [57.0510934286449]
  Training Large Language Models (LLMs) from scratch requires immense computational resources, making it prohibitively expensive. Model scaling-up offers a promising solution by leveraging the parameters of smaller models to create larger ones. We propose textbfLESA, a novel learnable method for depth scaling-up.
  arXiv  Detail & Related papers  (2025-02-19T14:58:48Z)
• Multi-Objective Large Language Model Unlearning [3.372396620898397]
  Gradient Ascent (GA) is a proactive way to decrease the prediction probability of the model on the target data. We propose Multi-Objective Large Language Model Unlearning (MOLLM) algorithm to overcome gradient explosion and catastrophic forgetting. Our empirical results verify that MoLLM outperforms the SOTA GA-based LLM unlearning methods in terms of unlearning effect and model utility preservation.
  arXiv  Detail & Related papers  (2024-12-29T09:35:56Z)
• FREE: Faster and Better Data-Free Meta-Learning [77.90126669914324]
  Data-Free Meta-Learning (DFML) aims to extract knowledge from a collection of pre-trained models without requiring the original data. We introduce the Faster and Better Data-Free Meta-Learning framework, which contains: (i) a meta-generator for rapidly recovering training tasks from pre-trained models; and (ii) a meta-learner for generalizing to new unseen tasks.
  arXiv  Detail & Related papers  (2024-05-02T03:43:19Z)
• Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning [55.96599486604344]
  We introduce an approach aimed at enhancing the reasoning capabilities of Large Language Models (LLMs) through an iterative preference learning process. We use Monte Carlo Tree Search (MCTS) to iteratively collect preference data, utilizing its look-ahead ability to break down instance-level rewards into more granular step-level signals. The proposed algorithm employs Direct Preference Optimization (DPO) to update the LLM policy using this newly generated step-level preference data.
  arXiv  Detail & Related papers  (2024-05-01T11:10:24Z)
• Task-Distributionally Robust Data-Free Meta-Learning [99.56612787882334]
  Data-Free Meta-Learning (DFML) aims to efficiently learn new tasks by leveraging multiple pre-trained models without requiring their original training data. For the first time, we reveal two major challenges hindering their practical deployments: Task-Distribution Shift ( TDS) and Task-Distribution Corruption (TDC)
  arXiv  Detail & Related papers  (2023-11-23T15:46:54Z)
• State Representation Learning Using an Unbalanced Atlas [8.938418994111716]
  This paper introduces a novel learning paradigm using an unbalanced atlas (UA), capable of surpassing state-of-the-art self-supervised learning approaches. The efficacy of DIM-UA is demonstrated through training and evaluation on the Atari Annotated RAM Interface benchmark.
  arXiv  Detail & Related papers  (2023-05-17T14:58:58Z)
• MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models [10.10825306582544]
  We propose MEta Loss TRansformer (MELTR), a plug-in module that automatically and non-linearly combines various loss functions to aid learning the target task via auxiliary learning. For evaluation, we apply our framework to various video foundation models (UniVL, Violet and All-in-one) and show significant performance gain on all four downstream tasks.
  arXiv  Detail & Related papers  (2023-03-23T03:06:44Z)
• Self-distillation with Online Diffusion on Batch Manifolds Improves Deep Metric Learning [23.974500845619175]
  We propose Online Batch Diffusion-based Self-Distillation (OBD-SD) for DML. We first propose a simple but effective Progressive Self-Distillation (PSD), which distills the knowledge progressively from the model itself during training. Then, we extend PSD with an Online Batch Diffusion Process (OBDP), which is to capture the local geometric structure of manifold in each batch.
  arXiv  Detail & Related papers  (2022-11-14T17:38:07Z)
• Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning [56.17603785248675]
  Model-agnostic meta-learning (MAML) has become a popular research area. Existing MAML algorithms rely on the episode' idea by sampling a few tasks and data points to update the meta-model at each iteration. This paper proposes memory-based algorithms for MAML that converge with vanishing error.
  arXiv  Detail & Related papers  (2021-06-09T08:47:58Z)
• Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification [59.326456778057384]
  We propose the Memory-based Multi-Source Meta-Learning framework to train a generalizable model for unseen domains. We also present a meta batch normalization layer (MetaBN) to diversify meta-test features. Experiments demonstrate that our M$3$L can effectively enhance the generalization ability of the model for unseen domains.
  arXiv  Detail & Related papers  (2020-12-01T11:38:16Z)
• Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation [4.1799778475823315]
  We propose a framework to enhance performance by meta-learning the initial conditions of existing DA algorithms. We present variants for both multi-source unsupervised domain adaptation (MSDA), and semi-supervised domain adaptation (SSDA) We achieve state of the art results on several DA benchmarks including the largest scale DomainNet.
  arXiv  Detail & Related papers  (2020-04-09T07:48:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.