Open Horizons: Evaluating Deep Models in the Wild

• URL: http://arxiv.org/abs/2512.12146v1
• Date: Sat, 13 Dec 2025 03:03:59 GMT
• Title: Open Horizons: Evaluating Deep Models in the Wild
• Abstract summary: We present a unified experimental study spanning open-set recognition (OSR) and few-shot class-incremental learning (FSCIL) on CIFAR-10.<n>For OSR, we compare three pretrained frozen visual encoders: ResNet-50, ConvNeXt-Tiny and CLIP ViT-B/16,using a linear probe and four post-hoc scoring functions, namely MSP, Energy, Mahalanobis and kNN.<n>For FSCIL, we compare modified SPPR, OrCo, and ConCM using partially frozen ResNet-50 across 1-, 5-, and 10-shot scenarios
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open-world deployment requires models to recognize both known categories and remain reliable when novel classes appear. We present a unified experimental study spanning open-set recognition (OSR) and few-shot class-incremental learning (FSCIL) on CIFAR-10. For OSR, we compare three pretrained frozen visual encoders: ResNet-50, ConvNeXt-Tiny and CLIP ViT-B/16,using a linear probe and four post-hoc scoring functions, namely MSP, Energy, Mahalanobis and kNN. Across metrics,such as, AUROC, AUPR, FPR@95, and OSCR, CLIP consistently yields the strongest separability between known and unknown samples, with Energy providing the most stable performance across backbones. For FSCIL, we compare modified SPPR, OrCo, and ConCM using partially frozen ResNet-50 across 1-, 5-, and 10-shot scenarios. ConCM achieves 84.7% accuracy in the 10-shot setting with the cleanest confusion matrix, while all methods show saturation beyond 5 shots. Our controlled evaluation reveals how the backbone architecture and scoring mechanisms affect unknown detection and how prototype-based methods mitigate catastrophic forgetting during incremental adaptation.

Related papers

• CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation [49.0564416580726]
  We distill the compositional judgments of a cross-attentive reranker into an embedding model.<n>We show that CORE-RERANKER-8B achieves an 82.7% total average, outperforming Jina-Reranker by 10.7 points.<n>The improvements transfer to the MCMR benchmark without sacrificing retrieval performance on COCO and Flickr30K.
  arXiv  Detail & Related papers  (2026-09-03T16:50:29Z)
• P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation [73.147281449267]
  We propose P2Fusion, a prior-guided distillation-based framework that reformulates infrared-visible image fusion.<n>Instead of imposing hard-coded penalties, we distill image-intrinsic priors, thermal saliency and spatial quality, into learnable dynamic regulators.<n>P2Fusion achieves state-of-the-art results in 14 out of 20 key evaluation metrics across 5 benchmarks.
  arXiv  Detail & Related papers  (2026-08-13T10:09:49Z)
• Multimodal Evaluator Preference Collapse: Cross-Modal Contagion in Self-Evolving Agents [1.0152838128195467]
  We show that Evaluator Preference Collapse ( EPC) is dramatically amplified in multimodal settings.<n>We then demonstrate a novel phenomenon we term cross-modal contagion.<n>We identify cross-model evaluator architecture as the primary risk factor for preference drift.
  arXiv  Detail & Related papers  (2026-06-15T13:18:20Z)
• Do Transformers Actually Help Intrusion Detection? A Temporal Sequence Evaluation on CIC-IDS2017 [1.2934180951771597]
  We reformulate CIC-IDS 2017 as a temporal intrusion-detection task by constructing ordered flow sequences from network conversations.<n>The central finding is that padding convention, not architecture, determines the Transformer's performance.<n>We advocate leakage-free splits, explicit padding disclosure, and sequence-aware benchmarking as standard practice in future IDS research.
  arXiv  Detail & Related papers  (2026-06-09T16:57:10Z)
• Open-Set Vein Biometric Recognition with Deep Metric Learning [0.0]
  We rigorously evaluate the computational boundaries of Deep Metric Learning (DML) under strict open-set constraints.<n>Our approach learns discnative L2-normalised embeddings and employs prototype-based matching with a similarity threshold.<n>On the large-scale MMCBNU 6000 benchmark, our best model (ResNet50-CBAM) achieves an OSCR of 0.9945, AUROC of 0.9974, and EER of 1.57%, maintaining high identification accuracy (99.6% Rank-1) while robustly rejecting unknown subjects.
  arXiv  Detail & Related papers  (2026-04-16T11:03:11Z)
• A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shifts [0.5156484100374058]
  We present a systematic comparison of out-of-distribution (OOD) detection methods across CLIP-stratified regimes.<n>Experiments cover two representation paradigms: CNNs trained from scratch and a fine-tuned Vision Transformer (ViT)<n>We find that the learned feature space largely determines OOD efficacy.
  arXiv  Detail & Related papers  (2025-11-14T23:18:13Z)
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• KoALA: KL-L0 Adversarial Detector via Label Agreement [9.360893727306324]
  KoALA is a semantics-free adversarial detector for deep neural networks.<n>It detects adversarial attacks when class predictions from two complementary similarity metrics disagree.<n>KoALA achieves a precision of 0.94 and a recall of 0.81 on ResNet/CIFAR-10, and a precision of 0.66 and a recall of 0.85 on CLIP/Tiny-ImageNet.
  arXiv  Detail & Related papers  (2025-10-14T17:33:00Z)
• COOkeD: Ensemble-based OOD detection in the era of zero-shot CLIP [47.84776775118222]
  Out-of-distribution (OOD) detection is an important building block in trustworthy image recognition systems.<n>We show that given a little open-mindedness from both ends, remarkable OOD detection can be achieved by instead creating a heterogeneous ensemble.<n>COkeD achieves state-of-the-art performance and greater robustness compared to both classical and CLIP-based OOD detection methods.
  arXiv  Detail & Related papers  (2025-07-30T11:02:38Z)
• Replay to Remember (R2R): An Efficient Uncertainty-driven Unsupervised Continual Learning Framework Using Generative Replay [1.5267291767316298]
  Continual Learning entails progressively acquiring knowledge from new data while retaining previously acquired knowledge.<n>We present a novel uncertainty-driven Unsupervised Continual Learning framework using Generative Replay, namely Replay to Remember (R2R)''<n>Our proposed R2R approach improves knowledge retention, achieving a state-of-the-art performance of 98.13%, 73.06%, 93.41%, 95.18%, 59.74%, respectively.
  arXiv  Detail & Related papers  (2025-05-07T20:29:31Z)
• CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classification [65.46685389276443]
  We ground our work on CLIP, a vision-language pre-trained encoder model that can perform zero-shot classification by matching an image with text prompts.<n>We then formulate purification risk as the KL divergence between the joint distributions purification process.<n>We propose two variants for our CLIPure approach: CLI-Diff which models the likelihood of images' latent vectors, and CLIPure-Cos which models the likelihood with the cosine similarity between the embeddings of an image and a photo of a.''
  arXiv  Detail & Related papers  (2025-02-25T13:09:34Z)
• GHOST: Gaussian Hypothesis Open-Set Technique [10.426399605773083]
  Evaluations of large-scale recognition methods typically focus on overall performance.<n> addressing fairness in Open-Set Recognition (OSR), we demonstrate that per-class performance can vary dramatically.<n>We apply Z-score normalization to logits to mitigate the impact of feature magnitudes that deviate from the model's expectations.
  arXiv  Detail & Related papers  (2025-02-05T16:56:14Z)
• Disentangle Your Dense Object Detector [82.22771433419727]
  Deep learning-based dense object detectors have achieved great success in the past few years and have been applied to numerous multimedia applications such as video understanding. However, the current training pipeline for dense detectors is compromised to lots of conjunctions that may not hold. We propose Disentangled Dense Object Detector (DDOD), in which simple and effective disentanglement mechanisms are designed and integrated into the current state-of-the-art detectors.
  arXiv  Detail & Related papers  (2021-07-07T00:52:16Z)
• Boosting Randomized Smoothing with Variance Reduced Classifiers [4.110108749051657]
  We motivate why ensembles are a particularly suitable choice as base models for Randomized Smoothing (RS) We empirically confirm this choice, obtaining state of the art results in multiple settings.
  arXiv  Detail & Related papers  (2021-06-13T08:40:27Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)
• CREPO: An Open Repository to Benchmark Credal Network Algorithms [78.79752265884109]
  Credal networks are imprecise probabilistic graphical models based on, so-called credal, sets of probability mass functions. A Java library called CREMA has been recently released to model, process and query credal networks. We present CREPO, an open repository of synthetic credal networks, provided together with the exact results of inference tasks on these models.
  arXiv  Detail & Related papers  (2021-05-10T07:31:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.