CAMO: A Class-Aware Minority-Optimized Ensemble for Robust Language Model Evaluation on Imbalanced Data

• URL: http://arxiv.org/abs/2604.07583v2
• Date: Sat, 11 Apr 2026 19:50:11 GMT
• Title: CAMO: A Class-Aware Minority-Optimized Ensemble for Robust Language Model Evaluation on Imbalanced Data
• Abstract summary: We provide a unique ensemble technique for imbalanced problems called CAMO (Class-Aware Minority-d)<n>Through a hierarchical procedure that incorporates vote distributions, confidence calibration, and inter model uncertainty, CAMO dynamically boosts underrepresented classes while preserving and amplifying minority forecasts.<n>We benchmark against seven proven ensemble algorithms using eight different language models (three LLMs and five SLMs) under zero-shot and fine-tuned settings.
• Score: 0.5097809301149341
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world categorization is severely hampered by class imbalance because traditional ensembles favor majority classes, which lowers minority performance and overall F1-score. We provide a unique ensemble technique for imbalanced problems called CAMO (Class-Aware Minority-Optimized).Through a hierarchical procedure that incorporates vote distributions, confidence calibration, and inter model uncertainty, CAMO dynamically boosts underrepresented classes while preserving and amplifying minority forecasts. We verify CAMO on two highly unbalanced, domain-specific benchmarks: the DIAR-AI/Emotion dataset and the ternary BEA 2025 dataset. We benchmark against seven proven ensemble algorithms using eight different language models (three LLMs and five SLMs) under zero-shot and fine-tuned settings .With refined models, CAMO consistently earns the greatest strict macro F1-score, setting a new benchmark. Its benefit works in concert with model adaptation, showing that the best ensemble choice depends on model properties .This proves that CAMO is a reliable, domain-neutral framework for unbalanced categorization.

Related papers

• Evaluation of AutoML Frameworks for IDS under Imbalanced Data Conditions of the NSL-KDD Dataset [0.0]
  This work investigates the impact of severe class imbalance on the performance of automated machine learning (AutoML) frameworks for network intrusion detection.<n>We preserve the original five-class distribution, including highly underrepresented attacks such as R2L and U2R.<n>Nine open-source AutoML frameworks were analyzed under a unified and reproducible experimental protocol.
  arXiv  Detail & Related papers  (2026-06-10T19:08:25Z)
• Diversity Matters: Revisiting Test-Time Compute in Vision-Language Models [60.9932576020062]
  Test-time compute strategies have emerged as a lightweight approach to boost reasoning in large language models.<n>We propose Entropy-based TTC, which selects the most confident prediction based on predictive entropy.<n>We prove that ETTC outperforms majority voting under mild assumptions and empirically demonstrate that it consistently surpasses both voting and the best individual model.
  arXiv  Detail & Related papers  (2026-05-29T01:06:38Z)
• Learning to Trust the Crowd: A Multi-Model Consensus Reasoning Engine for Large Language Models [0.0]
  Large language models (LLMs) achieve strong aver- age performance yet remain unreliable at the instance level.<n>We introduce a Multi-Model Consensus Reasoning Engine that treats the set of LLM outputs as input to a supervised meta-learner.<n>The system maps natural language responses into structured features using semantic embeddings, pairwise similarity and clustering statistics, lexical and structural cues, reasoning-quality scores, confidence estimates, and model-specific priors.
  arXiv  Detail & Related papers  (2026-01-12T06:27:06Z)
• Majority Rules: LLM Ensemble is a Winning Approach for Content Categorization [0.0]
  This study introduces an ensemble framework for unstructured text categorization using large language models (LLMs)<n>By integrating multiple models, the ensemble large language model (eLLM) framework addresses common weaknesses of individual systems.<n>eLLM achieves near human-expert-level performance, offering a scalable and reliable solution for taxonomy-based classification.
  arXiv  Detail & Related papers  (2025-11-11T05:10:09Z)
• Unbiased Max-Min Embedding Classification for Transductive Few-Shot Learning: Clustering and Classification Are All You Need [83.10178754323955]
  Few-shot learning enables models to generalize from only a few labeled examples.<n>We propose the Unbiased Max-Min Embedding Classification (UMMEC) Method, which addresses the key challenges in few-shot learning.<n>Our method significantly improves classification performance with minimal labeled data, advancing the state-of-the-art in annotatedL.
  arXiv  Detail & Related papers  (2025-03-28T07:23:07Z)
• FILM: Framework for Imbalanced Learning Machines based on a new unbiased performance measure and a new ensemble-based technique [37.94431794242543]
  This research addresses the challenges of handling unbalanced datasets for binary classification tasks.<n>Standard evaluation metrics are often biased by the disproportionate representation of the minority class.<n>We propose a novel metric, the Unbiased Integration Coefficients, which exhibits significantly reduced bias.
  arXiv  Detail & Related papers  (2025-03-06T12:15:56Z)
• Ensemble Methods for Sequence Classification with Hidden Markov Models [8.241486511994202]
  We present a lightweight approach to sequence classification using Ensemble Methods for Hidden Markov Models (HMMs) HMMs offer significant advantages in scenarios with imbalanced or smaller datasets due to their simplicity, interpretability, and efficiency. Our ensemble-based scoring method enables the comparison of sequences of any length and improves performance on imbalanced datasets.
  arXiv  Detail & Related papers  (2024-09-11T20:59:32Z)
• Universal Semi-supervised Model Adaptation via Collaborative Consistency Training [92.52892510093037]
  We introduce a realistic and challenging domain adaptation problem called Universal Semi-supervised Model Adaptation (USMA) We propose a collaborative consistency training framework that regularizes the prediction consistency between two models. Experimental results demonstrate the effectiveness of our method on several benchmark datasets.
  arXiv  Detail & Related papers  (2023-07-07T08:19:40Z)
• GREAT Score: Global Robustness Evaluation of Adversarial Perturbation using Generative Models [60.48306899271866]
  We present a new framework, called GREAT Score, for global robustness evaluation of adversarial perturbation using generative models. We show high correlation and significantly reduced cost of GREAT Score when compared to the attack-based model ranking on RobustBench. GREAT Score can be used for remote auditing of privacy-sensitive black-box models.
  arXiv  Detail & Related papers  (2023-04-19T14:58:27Z)
• CLIPood: Generalizing CLIP to Out-of-Distributions [73.86353105017076]
  Contrastive language-image pre-training (CLIP) models have shown impressive zero-shot ability, but the further adaptation of CLIP on downstream tasks undesirably degrades OOD performances. We propose CLIPood, a fine-tuning method that can adapt CLIP models to OOD situations where both domain shifts and open classes may occur on unseen test data. Experiments on diverse datasets with different OOD scenarios show that CLIPood consistently outperforms existing generalization techniques.
  arXiv  Detail & Related papers  (2023-02-02T04:27:54Z)
• On Universal Black-Box Domain Adaptation [53.7611757926922]
  We study an arguably least restrictive setting of domain adaptation in a sense of practical deployment. Only the interface of source model is available to the target domain, and where the label-space relations between the two domains are allowed to be different and unknown. We propose to unify them into a self-training framework, regularized by consistency of predictions in local neighborhoods of target samples.
  arXiv  Detail & Related papers  (2021-04-10T02:21:09Z)
• Label-Imbalanced and Group-Sensitive Classification under Overparameterization [32.923780772605596]
  Label-imbalanced and group-sensitive classification seeks to appropriately modify standard training algorithms to optimize relevant metrics. We show that a logit-adjusted loss modification to standard empirical risk minimization might be ineffective in general. We show that our results extend naturally to binary classification with sensitive groups, thus treating the two common types of imbalances (label/group) in a unifying way.
  arXiv  Detail & Related papers  (2021-03-02T08:09:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.