Causally-Guided Diffusion for Stable Feature Selection

• URL: http://arxiv.org/abs/2603.20930v1
• Date: Sat, 21 Mar 2026 20:14:00 GMT
• Title: Causally-Guided Diffusion for Stable Feature Selection
• Abstract summary: We study feature selection from a stability perspective and introduce Causally-Guided Diffusion for Stable Feature Selection.<n>In CGDFS, we formalized feature selection as approximate posterior inference over feature subsets.<n>We evaluate CGDFS on open-source real-world datasets exhibiting distribution shifts.
• Score: 29.398494058613124
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Feature selection is fundamental to robust data-centric AI, but most existing methods optimize predictive performance under a single data distribution. This often selects spurious features that fail under distribution shifts. Motivated by principles from causal invariance, we study feature selection from a stability perspective and introduce Causally-Guided Diffusion for Stable Feature Selection (CGDFS). In CGDFS, we formalized feature selection as approximate posterior inference over feature subsets, whose posterior mass favors low prediction error and low cross-environment variance. Our framework combines three key insights: First, we formulate feature selection as stability-aware posterior sampling. Here, causal invariance serves as a soft inductive bias rather than explicit causal discovery. Second, we train a diffusion model as a learned prior over plausible continuous selection masks, combined with a stability-aware likelihood that rewards invariance across environments. This diffusion prior captures structural dependencies among features and enables scalable exploration of the combinatorially large selection space. Third, we perform guided annealed Langevin sampling that combines the diffusion prior with the stability objective, which yields a tractable, uncertainty-aware posterior inference that avoids discrete optimization and produces robust feature selections. We evaluate CGDFS on open-source real-world datasets exhibiting distribution shifts. Across both classification and regression tasks, CGDFS consistently selects more stable and transferable feature subsets, which leads to improved out-of-distribution performance and greater selection robustness compared to sparsity-based, tree-based, and stability-selection baselines.

Related papers

• Preference Data Selection for Mitigating the Alignment Tax in Large Language Models [52.18310687523229]
  BALIGN is a balanced data selection strategy that mitigates catastrophic forgetting while optimizing alignment efficacy.<n>By aggregating three key data-centric features into a unified composite risk score, BALIGN systematically filters out high-risk preference samples.<n>Experiments on standard human preference datasets demonstrate that BALIGN strongly preserves foundational capabilities without compromising alignment gains.
  arXiv  Detail & Related papers  (2026-08-25T07:58:42Z)
• Freeze, Then Select: Structured Field Adapters and Stability-Validated Weak Selection for PDE Discovery from Sparse Observations [5.191092845381431]
  PDE discovery from sparse observations requires reconstructing a continuous field and selecting the correct differential terms.<n>We develop a freeze-then-select method combining a structured field adapter with Stability-d Weak Selection.<n>Our method attains the highest exact support recovery rate, with its clearest gains over classical and neural baselines on challenging Kuramoto-Sivashinsky dynamics.
  arXiv  Detail & Related papers  (2026-07-31T17:49:32Z)
• Traits Run Deeper: Trait-Specific Asymmetric Fusion for Personality Assessment [53.50824432699408]
  Traits Run Deeper is a novel personality assessment framework.<n>It consists of three components: the Multimodal Foundation Representation (MFR), the Trait-Specific Modality Fusion (TSMF) and the Distribution-Calibrated Personality Regression (DCPR)
  arXiv  Detail & Related papers  (2026-06-09T06:38:36Z)
• Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models [52.48582333951919]
  We propose a dynamic framework designed to enhance alignment reliability by maximizing the Signal-to-Noise Ratio of policy updates.<n>SAGE (Stability-Aware Gradient Efficiency) integrates a coarse-grained curriculum mechanism that refreshes candidate pools based on model competence.<n> Experiments on multiple mathematical reasoning benchmarks demonstrate that SAGE significantly accelerates convergence and outperforms static baselines.
  arXiv  Detail & Related papers  (2026-02-01T12:56:10Z)
• Decoupling the Effect of Chain-of-Thought Reasoning: A Human Label Variation Perspective [60.45433515408158]
  We show that long Chain-of-Thought (CoT) serves as a decisive decision-maker for the top option but fails to function as a granular distribution calibrator for ambiguous tasks.<n>We observe a distinct "decoupled mechanism": while CoT improves distributional alignment, final accuracy is dictated by CoT content.
  arXiv  Detail & Related papers  (2026-01-06T16:26:40Z)
• Latent Variable Causal Discovery under Selection Bias [31.127374830119454]
  We study rank constraints, which as a generalization to conditional independence constraints, exploits the ranks of covariance submatrices in linear Gaussian models.<n>We show that although selection can significantly complicate the joint distribution, the ranks in the biased covariance matrices still preserve meaningful information about both causal structures and selection mechanisms.<n>Using this tool, we demonstrate that the one-factor model, a classical latent variable model, can be identified under selection bias.
  arXiv  Detail & Related papers  (2025-12-12T02:00:01Z)
• TangledFeatures: Robust Feature Selection in Highly Correlated Spaces [0.0]
  We introduce TangledFeatures, a framework for feature selection in correlated feature spaces.<n>It identifies representative features from groups of entangled predictors, reducing redundancy while retaining explanatory power.<n>We show that the selected features correspond to structurally meaningful intra-atomic distances that explain variation in backbone torsional angles.
  arXiv  Detail & Related papers  (2025-10-16T05:54:04Z)
• Diffusion-Driven High-Dimensional Variable Selection [6.993247097440294]
  We propose a resample-aggregate framework that exploits diffusion models' ability to generate high-fidelity synthetic data.<n>We show that the proposed method is selection consistent under mild assumptions.<n>Our method advances variable selection methodology and broadens the toolkit for interpretable, statistically rigorous analysis.
  arXiv  Detail & Related papers  (2025-08-19T14:54:20Z)
• Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations [67.35596444651037]
  Vision-language models (VLMs) exhibit remarkable zero-shot capabilities but struggle with distribution shifts in downstream tasks when labeled data is unavailable.<n>We propose a Reliable Test-time Adaptation (ReTA) method that enhances reliability from two perspectives.
  arXiv  Detail & Related papers  (2025-07-13T05:37:33Z)
• Feature-Based Instance Neighbor Discovery: Advanced Stable Test-Time Adaptation in Dynamic World [7.9346432903946535]
  Deep neural networks still suffer performance declines under distribution shifts between training and test domains.<n>We propose Feature-based Instance Neighbor Discovery (FIND), which comprises three key components: Layer-wise Feature Disentanglement (LFD), Feature Aware Batch Normalization (FABN) and Selective FABN (S-FABN)<n>Experiments demonstrate FIND significantly outperforms existing methods, achieving a 30% accuracy improvement in dynamic scenarios.
  arXiv  Detail & Related papers  (2025-06-07T12:45:49Z)
• Stable Neighbor Denoising for Source-free Domain Adaptive Segmentation [91.83820250747935]
  Pseudo-label noise is mainly contained in unstable samples in which predictions of most pixels undergo significant variations during self-training. We introduce the Stable Neighbor Denoising (SND) approach, which effectively discovers highly correlated stable and unstable samples. SND consistently outperforms state-of-the-art methods in various SFUDA semantic segmentation settings.
  arXiv  Detail & Related papers  (2024-06-10T21:44:52Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• Loss-guided Stability Selection [0.0]
  It is well-known that model selection procedures like the Lasso or Boosting tend to overfit on real data. Standard Stability Selection is based on a global criterion, namely the per-family error rate. We propose a Stability Selection variant which respects the chosen loss function via an additional validation step.
  arXiv  Detail & Related papers  (2022-02-10T11:20:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.