Improving Arabic Multi-Label Emotion Classification using Stacked Embeddings and Hybrid Loss Function

• URL: http://arxiv.org/abs/2410.03979v3
• Date: Thu, 14 Nov 2024 14:34:13 GMT
• Title: Improving Arabic Multi-Label Emotion Classification using Stacked Embeddings and Hybrid Loss Function
• Authors: Muhammad Azeem Aslam, Wang Jun, Nisar Ahmed, Muhammad Imran Zaman, Li Yanan, Hu Hongfei, Wang Shiyu, Xin Liu,
• Abstract summary: This study uses stacked embeddings, meta-learning, and a hybrid loss function to enhance multi-label emotion classification for the Arabic language. To further improve performance, a hybrid loss function is introduced, incorporating class weighting, label correlation, and contrastive learning. Experiments validate the proposed model's performance across key metrics such as Precision, Recall, F1-Score, Jaccard Accuracy, and Hamming Loss.
• Score: 4.149971421068989
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In multi-label emotion classification, particularly for low-resource languages like Arabic, the challenges of class imbalance and label correlation hinder model performance, especially in accurately predicting minority emotions. To address these issues, this study proposes a novel approach that combines stacked embeddings, meta-learning, and a hybrid loss function to enhance multi-label emotion classification for the Arabic language. The study extracts contextual embeddings from three fine-tuned language models-ArabicBERT, MarBERT, and AraBERT-which are then stacked to form enriched embeddings. A meta-learner is trained on these stacked embeddings, and the resulting concatenated representations are provided as input to a Bi-LSTM model, followed by a fully connected neural network for multi-label classification. To further improve performance, a hybrid loss function is introduced, incorporating class weighting, label correlation matrix, and contrastive learning, effectively addressing class imbalances and improving the handling of label correlations. Extensive experiments validate the proposed model's performance across key metrics such as Precision, Recall, F1-Score, Jaccard Accuracy, and Hamming Loss. The class-wise performance analysis demonstrates the hybrid loss function's ability to significantly reduce disparities between majority and minority classes, resulting in a more balanced emotion classification. An ablation study highlights the contribution of each component, showing the superiority of the model compared to baseline approaches and other loss functions. This study not only advances multi-label emotion classification for Arabic but also presents a generalizable framework that can be adapted to other languages and domains, providing a significant step forward in addressing the challenges of low-resource emotion classification tasks.

Related papers

• Flick: Few Labels Text Classification using K-Aware Intermediate Learning in Multi-Task Low-Resource Languages [15.409164660580362]
  We propose Flick to address the persistent challenge of few-label text classification in truly low-resource linguistic contexts.<n>Flick learns to distil highly reliable pseudo-labels from an initial broad set by focusing on single-cluster cohesion and leveraging an adaptive top-k selection mechanism.<n>We demonstrate Flick's efficacy across 14 diverse datasets, encompassing challenging low-resource languages such as Arabic, Urdu, and Setswana.
  arXiv  Detail & Related papers  (2025-06-12T02:09:47Z)
• Class-Independent Increment: An Efficient Approach for Multi-label Class-Incremental Learning [49.65841002338575]
  This paper focuses on the challenging yet practical multi-label class-incremental learning (MLCIL) problem. We propose a novel class-independent incremental network (CINet) to extract multiple class-level embeddings for multi-label samples. It learns and preserves the knowledge of different classes by constructing class-specific tokens.
  arXiv  Detail & Related papers  (2025-03-01T14:40:52Z)
• Multi-Label Contrastive Learning : A Comprehensive Study [48.81069245141415]
  Multi-label classification has emerged as a key area in both research and industry. Applying contrastive learning to multi-label classification presents unique challenges. We conduct an in-depth study of contrastive learning loss for multi-label classification across diverse settings.
  arXiv  Detail & Related papers  (2024-11-27T20:20:06Z)
• Dual-Decoupling Learning and Metric-Adaptive Thresholding for Semi-Supervised Multi-Label Learning [81.83013974171364]
  Semi-supervised multi-label learning (SSMLL) is a powerful framework for leveraging unlabeled data to reduce the expensive cost of collecting precise multi-label annotations. Unlike semi-supervised learning, one cannot select the most probable label as the pseudo-label in SSMLL due to multiple semantics contained in an instance. We propose a dual-perspective method to generate high-quality pseudo-labels.
  arXiv  Detail & Related papers  (2024-07-26T09:33:53Z)
• Simple-Sampling and Hard-Mixup with Prototypes to Rebalance Contrastive Learning for Text Classification [11.072083437769093]
  We propose a novel model named SharpReCL for imbalanced text classification tasks. Our model even outperforms popular large language models across several datasets.
  arXiv  Detail & Related papers  (2024-05-19T11:33:49Z)
• Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversations [15.705757672984662]
  Multimodal Emotion Recognition in Conversations (MERC) is a significant development direction for machine intelligence. Many data in MERC naturally exhibit an imbalanced distribution of emotion categories, and researchers ignore the negative impact of imbalanced data on emotion recognition. We propose the Class Boundary Enhanced Representation Learning (CBERL) model to address the imbalanced distribution of emotion categories in raw data. We have conducted extensive experiments on the IEMOCAP and MELD benchmark datasets, and the results show that CBERL has achieved a certain performance improvement in the effectiveness of emotion recognition.
  arXiv  Detail & Related papers  (2023-12-11T12:35:17Z)
• PercentMatch: Percentile-based Dynamic Thresholding for Multi-Label Semi-Supervised Classification [64.39761523935613]
  We propose a percentile-based threshold adjusting scheme to dynamically alter the score thresholds of positive and negative pseudo-labels for each class during the training. We achieve strong performance on Pascal VOC2007 and MS-COCO datasets when compared to recent SSL methods.
  arXiv  Detail & Related papers  (2022-08-30T01:27:48Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• CLLD: Contrastive Learning with Label Distance for Text Classificatioin [0.6299766708197883]
  We propose Contrastive Learning with Label Distance (CLLD) for learning contrastive classes. CLLD ensures the flexibility within the subtle differences that lead to different label assignments. Our experiments suggest that the learned label distance relieve the adversarial nature of interclasses.
  arXiv  Detail & Related papers  (2021-10-25T07:07:14Z)
• Not All Negatives are Equal: Label-Aware Contrastive Loss for Fine-grained Text Classification [0.0]
  We analyse the contrastive fine-tuning of pre-trained language models on two fine-grained text classification tasks. We adaptively embed class relationships into a contrastive objective function to help differently weigh the positives and negatives. We find that Label-aware Contrastive Loss outperforms previous contrastive methods.
  arXiv  Detail & Related papers  (2021-09-12T04:19:17Z)
• Semi-Supervised Few-Shot Classification with Deep Invertible Hybrid Models [4.189643331553922]
  We propose a deep invertible hybrid model which integrates discriminative and generative learning at a latent space level for semi-supervised few-shot classification. Our main originality lies in our integration of these components at a latent space level, which is effective in preventing overfitting.
  arXiv  Detail & Related papers  (2021-05-22T05:55:16Z)
• Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection [69.2370349274216]
  Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances. Semantic components are distilled from utterances via multi-head self-attention. Our method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances.
  arXiv  Detail & Related papers  (2020-10-06T05:16:38Z)
• Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function [106.69643619725652]
  We develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results. We report state-of-the-art results for text classification task on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-09-08T21:55:22Z)
• Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification [52.69730591919885]
  We present a semi-supervised adversarial training process that minimizes the maximal loss for label-preserving input perturbations. We observe significant gains in effectiveness on document and intent classification for a diverse set of languages.
  arXiv  Detail & Related papers  (2020-07-29T19:38:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.