Structure-Adaptive Conformal Inference for Large-Scale Out-of-Distribution Testing

• URL: http://arxiv.org/abs/2605.26429v1
• Date: Tue, 26 May 2026 01:28:16 GMT
• Title: Structure-Adaptive Conformal Inference for Large-Scale Out-of-Distribution Testing
• Abstract summary: This paper addresses structured out-of-distribution (OOD) testing in high-stakes machine learning applications.<n>We propose the structure-adaptive conformal q-value (SCQ) and pseudo-score-guided automated model selection (P-TAMS)<n>P-TAMS adapts conformalized model selection to structured OOD testing across a toolbox of candidate models.
• Score: 1.867982979635437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper addresses structured out-of-distribution (OOD) testing in high-stakes machine learning applications. Traditional conformal methods rely on joint exchangeability, making it difficult to incorporate auxiliary information such as spatiotemporal or grouping structures. To overcome this limitation, we propose the structure-adaptive conformal q-value (SCQ), a significance index that integrates individual test evidence with structural patterns. We also develop pseudo-score-guided transductive automated model selection (P-TAMS), which adapts conformalized model selection to structured OOD testing across a toolbox of candidate models. Together, SCQ and P-TAMS form a unified framework under pairwise exchangeability, providing finite-sample error-rate control, improved power, and enhanced interpretability. Experiments on simulated and real data demonstrate that the proposed approach controls the false discovery rate and performs well across diverse settings.

Related papers

• Persistent Convolution: A Topological Framework for AI Alignment Testing and Semantic Space Characterization [0.0]
  This work develops and demonstrates a topology-based multi-modal alignment test to make deployment, selection, and comparison of opaque models more interpretable.<n>These methods also offer an intuitive connection to possibility theory and a unified decision theoretic framework from data to deployment.
  arXiv  Detail & Related papers  (2026-07-31T04:12:21Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration [72.0672328514289]
  Multimodal learning often grapples with the challenge of low-quality data.<n>We propose a unified framework, termed Conformal Predictive Self-Calibration.<n>Our framework consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-05T14:48:52Z)
• An Adaptive, Data-Integrated Agent-Based Modeling Framework for Explainable and Contestable Policy Design [0.0]
  Multi-agent systems often operate under feedback, adaptation, and non-stationarity.<n>This paper introduces a general adaptive multi-agent learning framework.
  arXiv  Detail & Related papers  (2025-11-24T21:41:45Z)
• Adapformer: Adaptive Channel Management for Multivariate Time Series Forecasting [49.40321003932633]
  Adapformer is an advanced Transformer-based framework that merges the benefits of CI and CD methodologies through effective channel management.<n>Adapformer achieves superior performance over existing models, enhancing both predictive accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2025-11-18T16:24:05Z)
• Open-World Test-Time Adaptation with Hierarchical Feature Aggregation and Attention Affine [17.151364853811128]
  Test-time adaptation (TTA) refers to adjusting the model during the testing phase to cope with changes in sample distribution.<n>We propose a Hierarchical Ladder Network that extracts OOD features from class tokens aggregated across all Transformer layers.<n>We also introduce an Attention Affine Network (AAN) that adaptively refines the self-attention mechanism conditioned on the token information to better adapt to domain drift.
  arXiv  Detail & Related papers  (2025-11-16T14:05:23Z)
• An Integrated Fusion Framework for Ensemble Learning Leveraging Gradient Boosting and Fuzzy Rule-Based Models [59.13182819190547]
  Fuzzy rule-based models excel in interpretability and have seen widespread application across diverse fields.<n>They face challenges such as complex design specifications and scalability issues with large datasets.<n>This paper proposes an Integrated Fusion Framework that merges the strengths of both paradigms to enhance model performance and interpretability.
  arXiv  Detail & Related papers  (2025-11-11T10:28:23Z)
• Unified modality separation: A vision-language framework for unsupervised domain adaptation [60.8391821117794]
  Unsupervised domain adaptation (UDA) enables models trained on a labeled source domain to handle new unlabeled domains.<n>We propose a unified modality separation framework that accommodates both modality-specific and modality-invariant components.<n>Our methods achieve up to 9% performance gain with 9 times of computational efficiencies.
  arXiv  Detail & Related papers  (2025-08-07T02:51:10Z)
• Score-based Generative Modeling for Conditional Independence Testing [35.0533359302886]
  We propose a novel CI testing method via score-based generative modeling, which achieves precise Type I error control and strong testing power.<n>We theoretically establish the error bound of conditional distributions modeled by score-based generative models and prove the validity of our CI tests.
  arXiv  Detail & Related papers  (2025-05-29T10:10:46Z)
• A SCADE Model Verification Method Based on B-Model Transformation [0.8437187555622164]
  This study proposes a formal verification framework based on the B-Method.<n>It successfully verifies abstract specifications that are difficult to model directly in SCADE.<n>This study provides a cross-model verification paradigm for embedded control systems in avionics, rail transportation, and other domains.
  arXiv  Detail & Related papers  (2025-05-02T03:05:09Z)
• SKADA-Bench: Benchmarking Unsupervised Domain Adaptation Methods with Realistic Validation On Diverse Modalities [50.6382396309597]
  Unsupervised Domain Adaptation (DA) consists of adapting a model trained on a labeled source domain to perform well on an unlabeled target domain with some data distribution shift.<n>We present a complete and fair evaluation of existing shallow algorithms, including reweighting, mapping, and subspace alignment.<n>Our benchmark highlights the importance of realistic validation and provides practical guidance for real-life applications.
  arXiv  Detail & Related papers  (2024-07-16T12:52:29Z)
• Structured Conformal Inference for Matrix Completion with Applications to Group Recommender Systems [16.519348575982004]
  We develop a conformal inference method to construct a joint confidence region for a given group of missing entries within a sparsely observed matrix.<n>Our method is model-agnostic and can be combined with any black-box'' matrix completion algorithm to provide reliable uncertainty estimation for group-level recommendations.
  arXiv  Detail & Related papers  (2024-04-26T17:42:29Z)
• A New Score for Adaptive Tests in Bayesian and Credal Networks [64.80185026979883]
  A test is adaptive when its sequence and number of questions is dynamically tuned on the basis of the estimated skills of the taker. We present an alternative family of scores, based on the mode of the posterior probabilities, and hence easier to explain.
  arXiv  Detail & Related papers  (2021-05-25T20:35:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.