Towards Macro-AUC oriented Imbalanced Multi-Label Continual Learning

• URL: http://arxiv.org/abs/2412.18231v1
• Date: Tue, 24 Dec 2024 07:30:20 GMT
• Title: Towards Macro-AUC oriented Imbalanced Multi-Label Continual Learning
• Authors: Yan Zhang, Guoqiang Wu, Bingzheng Wang, Teng Pang, Haoliang Sun, Yilong Yin,
• Abstract summary: In Continual Learning (CL), while existing work primarily focuses on the multi-class classification task, there has been limited research on Multi-Label Learning (MLL)<n>In this paper, we propose a new memory replay-based method to tackle the imbalance issue for Macro-AUC-oriented MLCL.
• Score: 32.7137109236055
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In Continual Learning (CL), while existing work primarily focuses on the multi-class classification task, there has been limited research on Multi-Label Learning (MLL). In practice, MLL datasets are often class-imbalanced, making it inherently challenging, a problem that is even more acute in CL. Due to its sensitivity to imbalance, Macro-AUC is an appropriate and widely used measure in MLL. However, there is no research to optimize Macro-AUC in MLCL specifically. To fill this gap, in this paper, we propose a new memory replay-based method to tackle the imbalance issue for Macro-AUC-oriented MLCL. Specifically, inspired by recent theory work, we propose a new Reweighted Label-Distribution-Aware Margin (RLDAM) loss. Furthermore, to be compatible with the RLDAM loss, a new memory-updating strategy named Weight Retain Updating (WRU) is proposed to maintain the numbers of positive and negative instances of the original dataset in memory. Theoretically, we provide superior generalization analyses of the RLDAM-based algorithm in terms of Macro-AUC, separately in batch MLL and MLCL settings. This is the first work to offer theoretical generalization analyses in MLCL to our knowledge. Finally, a series of experimental results illustrate the effectiveness of our method over several baselines. Our codes are available at https://github.com/ML-Group-SDU/Macro-AUC-CL.

Related papers

• Mitigating Modality Imbalance in Multi-modal Learning via Multi-objective Optimization [57.00656508727821]
  Multi-modal learning (MML) aims to integrate information from multiple modalities, which is expected to lead to superior performance over single-modality learning.<n>Recent studies have shown that MML can underperform, even compared to single-modality approaches, due to imbalanced learning across modalities.<n>We propose a gradient-based algorithm to solve the modified MML problem.
  arXiv  Detail & Related papers  (2025-11-10T04:16:01Z)
• Large Language Models as Universal Predictors? An Empirical Study on Small Tabular Datasets [0.0]
  Large Language Models (LLMs) can perform predictive tasks over structured inputs without explicit fine-tuning on downstream tasks.<n>We investigate the empirical function approximation capability of LLMs on small-scale structured datasets for classification, regression and clustering tasks.<n>Our findings suggest that LLMs can serve as general-purpose predictive engines for structured data, with clear strengths in classification and significant limitations in regression and clustering.
  arXiv  Detail & Related papers  (2025-08-24T15:00:51Z)
• Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMs [32.09688735548092]
  We propose an elegantly simple MCL paradigm called "MErge then ReAlign" (MERA) to address both forgetting and misalignment.<n>Experiments demonstrate the impressive performance of MERA, holding an average of 99.84% Backward Relative Gain when extending to four modalities.
  arXiv  Detail & Related papers  (2025-03-08T20:29:40Z)
• Multi-Agent LLMs Ensemble for Efficient Atrial Fibrillation Annotation of ECG Reports [10.319249991495004]
  This study introduces a novel multiagent ensemble method powered by LLMs to address a key challenge in ML - data labeling. By using the ensemble LLMs and natural language processing, we labeled MIMIC-IV ECG dataset of 623,566 ECG reports with an estimated accuracy of 98.2%. We applied the ensemble LLMs method to identify SDOH from social history sections of 1,405 EHR clinical notes, also achieving competitive performance.
  arXiv  Detail & Related papers  (2024-10-21T22:12:00Z)
• Improved Diversity-Promoting Collaborative Metric Learning for Recommendation [127.08043409083687]
  Collaborative Metric Learning (CML) has recently emerged as a popular method in recommendation systems. This paper focuses on a challenging scenario where a user has multiple categories of interests. We propose a novel method called textitDiversity-Promoting Collaborative Metric Learning (DPCML)
  arXiv  Detail & Related papers  (2024-09-02T07:44:48Z)
• Regression-aware Inference with LLMs [52.764328080398805]
  We show that an inference strategy can be sub-optimal for common regression and scoring evaluation metrics. We propose alternate inference strategies that estimate the Bayes-optimal solution for regression and scoring metrics in closed-form from sampled responses.
  arXiv  Detail & Related papers  (2024-03-07T03:24:34Z)
• Task Aware Modulation using Representation Learning: An Approach for Few Shot Learning in Environmental Systems [15.40286222692196]
  TAM-RL is a novel framework for few-shot learning in heterogeneous systems. We evaluate TAM-RL on two real-world environmental datasets.
  arXiv  Detail & Related papers  (2023-10-07T07:55:22Z)
• Towards Understanding Generalization of Macro-AUC in Multi-label Learning [48.015768048227166]
  We characterize the generalization properties of various learning algorithms based on Macro-AUC. We identify a critical factor of the dataset affecting the generalization bounds: emphthe label-wise class imbalance We propose a new (and more general) McDiarmid-type concentration inequality, which may be of independent interest.
  arXiv  Detail & Related papers  (2023-05-09T08:13:48Z)
• Learning in Imperfect Environment: Multi-Label Classification with Long-Tailed Distribution and Partial Labels [53.68653940062605]
  We introduce a novel task, Partial labeling and Long-Tailed Multi-Label Classification (PLT-MLC) We find that most LT-MLC and PL-MLC approaches fail to solve the degradation-MLC. We propose an end-to-end learning framework: textbfCOrrection $rightarrow$ textbfModificattextbfIon $rightarrow$ balantextbfCe.
  arXiv  Detail & Related papers  (2023-04-20T20:05:08Z)
• Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning [71.26635165491105]
  We develop a sharpness-aware MAML approach that we term Sharp-MAML. We empirically demonstrate that Sharp-MAML and its computation-efficient variant can outperform popular existing MAML baselines. This is the first empirical and theoretical study on sharpness-aware minimization in the context of bilevel learning.
  arXiv  Detail & Related papers  (2022-06-08T16:20:11Z)
• MAML is a Noisy Contrastive Learner [72.04430033118426]
  Model-agnostic meta-learning (MAML) is one of the most popular and widely-adopted meta-learning algorithms nowadays. We provide a new perspective to the working mechanism of MAML and discover that: MAML is analogous to a meta-learner using a supervised contrastive objective function. We propose a simple but effective technique, zeroing trick, to alleviate such interference.
  arXiv  Detail & Related papers  (2021-06-29T12:52:26Z)
• Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning [56.17603785248675]
  Model-agnostic meta-learning (MAML) has become a popular research area. Existing MAML algorithms rely on the episode' idea by sampling a few tasks and data points to update the meta-model at each iteration. This paper proposes memory-based algorithms for MAML that converge with vanishing error.
  arXiv  Detail & Related papers  (2021-06-09T08:47:58Z)
• Revisiting Training Strategies and Generalization Performance in Deep Metric Learning [28.54755295856929]
  We revisit the most widely used DML objective functions and conduct a study of the crucial parameter choices. Under consistent comparison, DML objectives show much higher saturation than indicated by literature. Exploiting these insights, we propose a simple, yet effective, training regularization to reliably boost the performance of ranking-based DML models.
  arXiv  Detail & Related papers  (2020-02-19T22:16:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.