Automatic Unsupervised Ensemble Outlier Model Selection--Extended Version

• URL: http://arxiv.org/abs/2605.16567v1
• Date: Fri, 15 May 2026 19:15:58 GMT
• Title: Automatic Unsupervised Ensemble Outlier Model Selection--Extended Version
• Abstract summary: Unsupervised outlier detection is attractive because it eliminates the need for labeled data.<n>We propose MetaEns, an unsupervised framework for selecting ensembles of outlier detection models.<n>Experiments on 39 real-world datasets show that MetaEns consistently outperforms state-of-the-art unsupervised selectors and ensemble baselines.
• Score: 16.177590208339176
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Unsupervised outlier detection is attractive because it eliminates the need for labeled data. Moreover, forming multi-model ensembles can improve detection robustness. However, composing an ensemble without labeled data is challenging. Naively composed ensembles can suffer from ensemble saturation, where redundant or unreliable detection models degrade performance and incur unnecessary computation. We propose MetaEns, an automatic unsupervised framework for selecting ensembles of outlier detection models. Using labeled meta-datasets, MetaEns learns a model that predicts marginal ensemble gains, estimating the expected improvement from adding a candidate model to a partially constructed ensemble. At test time, this learned signal is combined with a submodular-inspired proxy objective that enforces diminishing returns through diversity-aware discounting and family-level risk regularization, thereby enabling greedy sequential selection with adaptive early stopping. As a result, MetaEns constructs compact, high-quality ensembles without access to ground-truth labels. Experiments on 39 real-world datasets show that MetaEns consistently outperforms state-of-the-art unsupervised selectors and ensemble baselines, achieving higher average precision while using fewer models.

Related papers

• Lost in Aggregation: How Benchmarks Overlook Irreplaceable Model Strengths [23.84195042261017]
  We argue that benchmark evaluation should also consider the data-centric peak performance frontier.<n>Applying this framework to the TabArena benchmark, we find that common aggregation metrics are highly correlated.
  arXiv  Detail & Related papers  (2026-08-19T13:51:05Z)
• When Metrics Disagree: A Meta-Analysis of Knowledge-Graph-Completion Model Benchmarking [0.0]
  We present a meta-analysis of seven aggregators across five tests: consistency, cross-dataset stability, metric independence, robustness under noise, and generalizability.<n>Across tail $(h,r,?)$ and relation $(h,?,t)$ prediction, Z-score is the most balanced aggregator, which ranks DualE highest for tail prediction and FMS highest for relation prediction.<n>The framework resolves evaluation inconsistencies and offers evidence-based guidance for aggregator selection and model benchmarking in KGC.
  arXiv  Detail & Related papers  (2026-06-09T01:20:43Z)
• SetAD: Semi-Supervised Anomaly Learning in Contextual Sets [25.628827917857603]
  Semi-supervised anomaly detection has shown great promise by effectively leveraging limited labeled data.<n>We propose SetAD, a novel framework that reframes semi-supervised AD as a Set-level Anomaly Detection task.<n>To enhance robustness and score calibration, we propose a context-calibrated anomaly scoring mechanism.
  arXiv  Detail & Related papers  (2025-11-26T13:27:59Z)
• Generalization is not a universal guarantee: Estimating similarity to training data with an ensemble out-of-distribution metric [0.09363323206192666]
  Failure of machine learning models to generalize to new data is a core problem limiting the reliability of AI systems.<n>We propose a standardized approach for assessing data similarity by constructing a supervised autoencoder for generalizability estimation (SAGE)<n>We show that out-of-the-box model performance increases after SAGE score filtering, even when applied to data from the model's own training and test datasets.
  arXiv  Detail & Related papers  (2025-02-22T19:21:50Z)
• Ranked from Within: Ranking Large Multimodal Models Without Labels [73.96543593298426]
  We show that uncertainty scores derived from softmax distributions provide a robust basis for ranking models across various tasks.<n>This facilitates the ranking of LMMs on unlabeled data, providing a practical approach for selecting models for diverse target domains without requiring manual annotation.
  arXiv  Detail & Related papers  (2024-12-09T13:05:43Z)
• Trained Models Tell Us How to Make Them Robust to Spurious Correlation without Group Annotation [3.894771553698554]
  Empirical Risk Minimization (ERM) models tend to rely on attributes that have high spurious correlation with the target. This can degrade the performance on underrepresented (or'minority') groups that lack these attributes. We propose Environment-based Validation and Loss-based Sampling (EVaLS) to enhance robustness to spurious correlation.
  arXiv  Detail & Related papers  (2024-10-07T08:17:44Z)
• Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification [49.09505771145326]
  We propose a Hierarchical Dynamic Labeling (HDL) algorithm that does not depend on model predictions and utilizes image embeddings to generate sample labels. Our approach has the potential to change the paradigm of pseudo-label generation in semi-supervised learning.
  arXiv  Detail & Related papers  (2024-04-26T06:00:27Z)
• Unsupervised Model Selection for Time-series Anomaly Detection [7.8027110514393785]
  We identify three classes of surrogate (unsupervised) metrics, namely, prediction error, model centrality, and performance on injected synthetic anomalies. We formulate metric combination with multiple imperfect surrogate metrics as a robust rank aggregation problem. Large-scale experiments on multiple real-world datasets demonstrate that our proposed unsupervised approach is as effective as selecting the most accurate model.
  arXiv  Detail & Related papers  (2022-10-03T16:49:30Z)
• Generative Modeling Helps Weak Supervision (and Vice Versa) [87.62271390571837]
  We propose a model fusing weak supervision and generative adversarial networks. It captures discrete variables in the data alongside the weak supervision derived label estimate. It is the first approach to enable data augmentation through weakly supervised synthetic images and pseudolabels.
  arXiv  Detail & Related papers  (2022-03-22T20:24:21Z)
• Examining and Combating Spurious Features under Distribution Shift [94.31956965507085]
  We define and analyze robust and spurious representations using the information-theoretic concept of minimal sufficient statistics. We prove that even when there is only bias of the input distribution, models can still pick up spurious features from their training data. Inspired by our analysis, we demonstrate that group DRO can fail when groups do not directly account for various spurious correlations.
  arXiv  Detail & Related papers  (2021-06-14T05:39:09Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)
• Automating Outlier Detection via Meta-Learning [37.736124230543865]
  We develop the first principled data-driven approach to model selection for outlier detection, called MetaOD, based on meta-learning. We show the effectiveness of MetaOD in selecting a detection model that significantly outperforms the most popular outlier detectors. To foster and further research on this new problem, we open-source our entire meta-learning system, benchmark environment, and testbed datasets.
  arXiv  Detail & Related papers  (2020-09-22T15:14:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.