Theoretical Grounding of Out-Of-Distribution Detection With Reinforcement Learning Optimizer

• URL: http://arxiv.org/abs/2606.17477v1
• Date: Tue, 16 Jun 2026 03:40:03 GMT
• Title: Theoretical Grounding of Out-Of-Distribution Detection With Reinforcement Learning Optimizer
• Abstract summary: Out-of-distribution (OOD) detection in dynamic open-world environments requires a model to continually adapt to evolving data distributions.<n>Most existing OOD detection methods optimize only the current-step objective and do not explicitly account for how post-deployment environment changes affect future OOD behavior.
• Score: 3.4364890279370623
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Out-of-distribution (OOD) detection in dynamic open-world environments requires a model to continually adapt to evolving data distributions while generalizing to covariate-shifted inputs and rejecting semantic-shifted OOD examples. Most existing OOD detection methods optimize only the current-step objective and do not explicitly account for how post-deployment environment changes affect future OOD behavior. In this paper, we establish a theoretical grounding for dynamic OOD detection using a reinforcement learning (RL)-guided optimizer that explicitly favors updates that reduce the semantic OOD false positive rate over time. We develop a novel augmented optimizer that uses an RL-guided correction term on top of standard gradient descent (GD) and show its improvement over both future-domain generalization and semantic-OOD rejection. We analyze temporal error decomposition in terms of model-change and environment-change generalization errors and develop a new theoretical framework for comparing the generalization errors under both GD and RL-guided optimizers.

Related papers

• IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning [57.16513298507272]
  We propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR)<n>IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization.<n>For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional.
  arXiv  Detail & Related papers  (2026-08-11T08:20:02Z)
• $Δ\mathrm{Energy}$: Optimizing Energy Change During Vision-Language Alignment Improves both OOD Detection and OOD Generalization [43.79061207312421]
  We introduce a novel energy-based OOD score, named DeltaEnergy.<n>DeltaEnergy significantly outperforms the vanilla energy-based OOD score and provides a more reliable approach for OOD detection.<n>EBM is theoretically proven to not only enhance OOD detection but also yields a domain-consistent Hessian.
  arXiv  Detail & Related papers  (2025-10-13T11:36:58Z)
• Knowledge Regularized Negative Feature Tuning of Vision-Language Models for Out-of-Distribution Detection [54.433899174017185]
  Out-of-distribution (OOD) detection is crucial for building reliable machine learning models.<n>We propose a novel method called Knowledge Regularized Negative Feature Tuning (KR-NFT)<n>NFT applies distribution-aware transformations to pre-trained text features, effectively separating positive and negative features into distinct spaces.<n>When trained with few-shot samples from ImageNet dataset, KR-NFT not only improves ID classification accuracy and OOD detection but also significantly reduces the FPR95 by 5.44%.
  arXiv  Detail & Related papers  (2025-07-26T07:44:04Z)
• Divergence Minimization Preference Optimization for Diffusion Model Alignment [66.31417479052774]
  Divergence Minimization Preference Optimization (DMPO) is a principled method for aligning diffusion models by minimizing reverse KL divergence.<n>DMPO can consistently outperform or match existing techniques across different base models and test sets.
  arXiv  Detail & Related papers  (2025-07-10T07:57:30Z)
• Optimizing Latent Dimension Allocation in Hierarchical VAEs: Balancing Attenuation and Information Retention for OOD Detection [14.833454650943805]
  hierarchical variational autoencoders (HVAEs) offer improved representational capacity over traditional VAEs.<n>Existing approaches often allocate latent capacity arbitrarily, leading to ineffective representations or posterior collapse.<n>We introduce a theoretically grounded framework for optimizing latent dimension allocation in HVAEs.<n>We prove the existence of an optimal allocation ratio $rast$ under a fixed latent budget, and empirically show that tuning this ratio consistently improves OOD detection performance.
  arXiv  Detail & Related papers  (2025-06-11T18:16:19Z)
• Learning where to learn: Training data distribution optimization for scientific machine learning [4.726397480637032]
  In scientific machine learning, models are routinely deployed with parameter values or boundary conditions far from those used in training.<n>This paper studies the learning-where-to-learn problem of designing a training data distribution that minimizes average prediction error.
  arXiv  Detail & Related papers  (2025-05-27T18:00:58Z)
• CRoFT: Robust Fine-Tuning with Concurrent Optimization for OOD Generalization and Open-Set OOD Detection [42.33618249731874]
  We show that minimizing the magnitude of energy scores on training data leads to domain-consistent Hessians of classification loss. We have developed a unified fine-tuning framework that allows for concurrent optimization of both tasks.
  arXiv  Detail & Related papers  (2024-05-26T03:28:59Z)
• Towards Robust Out-of-Distribution Generalization Bounds via Sharpness [41.65692353665847]
  We study the effect of sharpness on how a model tolerates data change in domain shift. We propose a sharpness-based OOD generalization bound by taking robustness into consideration.
  arXiv  Detail & Related papers  (2024-03-11T02:57:27Z)
• Towards Calibrated Robust Fine-Tuning of Vision-Language Models [97.19901765814431]
  This work proposes a robust fine-tuning method that improves both OOD accuracy and confidence calibration simultaneously in vision language models. We show that both OOD classification and OOD calibration errors have a shared upper bound consisting of two terms of ID data. Based on this insight, we design a novel framework that conducts fine-tuning with a constrained multimodal contrastive loss enforcing a larger smallest singular value.
  arXiv  Detail & Related papers  (2023-11-03T05:41:25Z)
• On the Impact of Spurious Correlation for Out-of-distribution Detection [14.186776881154127]
  We present a new formalization and model the data shifts by taking into account both the invariant and environmental features. Our results suggest that the detection performance is severely worsened when the correlation between spurious features and labels is increased in the training set.
  arXiv  Detail & Related papers  (2021-09-12T23:58:17Z)
• Improved OOD Generalization via Adversarial Training and Pre-training [49.08683910076778]
  In this paper, we theoretically show that a model robust to input perturbations generalizes well on OOD data. Inspired by previous findings that adversarial training helps improve input-robustness, we show that adversarially trained models have converged excess risk on OOD data.
  arXiv  Detail & Related papers  (2021-05-24T08:06:35Z)
• CASTLE: Regularization via Auxiliary Causal Graph Discovery [89.74800176981842]
  We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features.
  arXiv  Detail & Related papers  (2020-09-28T09:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.