iLOCO: Distribution-Free Inference for Feature Interactions

• URL: http://arxiv.org/abs/2502.06661v2
• Date: Thu, 22 May 2025 18:55:47 GMT
• Title: iLOCO: Distribution-Free Inference for Feature Interactions
• Abstract summary: We develop a new model-agnostic metric for measuring the importance of pairwise feature interactions.<n>We also introduce an ensemble learning method for calculating the iLOCO metric and confidence intervals.<n>We validate our iLOCO metric and our confidence intervals on both synthetic and real data sets.
• Score: 4.56754610152086
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Feature importance measures are widely studied and are essential for understanding model behavior, guiding feature selection, and enhancing interpretability. However, many machine learning fitted models involve complex interactions between features. Existing feature importance metrics fail to capture these pairwise or higher-order effects, while existing interaction metrics often suffer from limited applicability or excessive computation; no methods exist to conduct statistical inference for feature interactions. To bridge this gap, we first propose a new model-agnostic metric, interaction Leave-One-Covariate-Out (iLOCO), for measuring the importance of pairwise feature interactions, with extensions to higher-order interactions. Next, we leverage recent advances in LOCO inference to develop distribution-free and assumption-light confidence intervals for our iLOCO metric. To address computational challenges, we also introduce an ensemble learning method for calculating the iLOCO metric and confidence intervals that we show is both computationally and statistically efficient. We validate our iLOCO metric and our confidence intervals on both synthetic and real data sets, showing that our approach outperforms existing methods and provides the first inferential approach to detecting feature interactions.

Related papers

• Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis [55.78822230833247]
  We present Inter-X++, a comprehensive and large-scale benchmark designed to empower versatile HHI analysis.<n>Inter-X++ provides 11,388 high-fidelity interaction sequences and over 8.1M frames, featuring precise whole-body movements and detailed finger articulations.<n>We propose OpenHHI, a single and unified HHI representation and modeling framework that jointly optimize interaction reconstruction and semantic understanding.
  arXiv  Detail & Related papers  (2026-08-20T17:51:48Z)
• Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wild [49.77540427384148]
  We present AgentHOI, a training-free, agentic framework that transfers the generalist multimodal reasoning capabilities of foundation models to HOI detection in the wild.<n>To address the challenges of incomplete interaction discovery and ambiguous localization in complex scenes, we introduce two key mechanisms.<n>AgentHOI achieves superior performance over state-of-the-art supervised and weakly supervised methods in real-world settings.
  arXiv  Detail & Related papers  (2026-07-15T14:30:20Z)
• Modeling Nonlinear Feature Interactions with Product-Unit Residual Networks [0.9367224590861915]
  Product-unit residual networks (PURe) integrate multiplicative product units with residual connections to explicitly model cross-feature couplings.<n>We conduct a systematic evaluation on an interaction-driven synthetic benchmark and two real-world datasets.<n> PURe achieves competitive or improved performance, better robustness and sample efficiency in low-data regimes, and enhanced interaction-level interpretability.
  arXiv  Detail & Related papers  (2026-06-05T03:11:08Z)
• Activation Functions, Statistics and Learning of Higher-Order Interactions in Restricted Boltzmann Machines [0.0]
  We study the statistics of the interactions induced by Restricted Boltzmann Machines with different hidden unit activation functions.<n>We characterize the space of representable models analytically in terms of moments of the distribution of induced interactions.<n>We find that rapidly increasing nonlinearities, such as the Exponential function, can facilitate the representation and learning of such data structures.
  arXiv  Detail & Related papers  (2026-05-18T23:04:24Z)
• A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactions [6.929805474293474]
  We propose a unified approach to explore the common mechanism of various KD methods using interactions.<n>We discover that the common mechanism underlying various KD methods is the sparsification of interactions.<n>Motivated by these insights, we propose a plug-and-play loss function called Complex Interaction Penalty (CIP) to explicitly enforce the sparsity of complex interactions.
  arXiv  Detail & Related papers  (2026-05-05T07:24:01Z)
• Zero-shot HOI Detection with MLLM-based Detector-agnostic Interaction Recognition [71.5328300638085]
  Zero-shot Human-object interaction (HOI) detection aims to locate humans and objects in images and recognize their interactions.<n>Existing methods, including two-stage methods, tightly couple interaction recognition with a specific detector.<n>We propose a decoupled framework that separates object detection from IR and leverages multi-modal large language models (MLLMs) for zero-shot IR.
  arXiv  Detail & Related papers  (2026-02-16T19:01:31Z)
• Explainability of Complex AI Models with Correlation Impact Ratio [10.61008729196936]
  Complex AI systems make better predictions but often lack transparency, limiting trustworthiness, interpretability, and safe deployment.<n>We introduce ExCIR (Explainability through Correlation Impact Ratio), a theoretically grounded, simple, and reliable metric for explaining the contribution of input features to model outputs.<n>We demonstrate that ExCIR captures dependencies arising from correlated features through a lightweight single pass formulation.
  arXiv  Detail & Related papers  (2026-01-10T21:56:24Z)
• Using Large Language Models to Detect Socially Shared Regulation of Collaborative Learning [15.567266973412815]
  We extend predictive models to automatically detect socially shared regulation of learning behaviors using embedding-based approaches.<n>We leverage large language models (LLMs) as summarization tools to generate task-aware representations of student dialogue aligned with system logs.<n>Results show that text-only embeddings often achieve stronger performance in detecting SSRL behaviors related to enactment or group dynamics.
  arXiv  Detail & Related papers  (2026-01-08T00:30:46Z)
• Federated Attention: A Distributed Paradigm for Collaborative LLM Inference over Edge Networks [63.541114376141735]
  Large language models (LLMs) are proliferating rapidly at the edge, delivering intelligent capabilities across diverse application scenarios.<n>However, their practical deployment in collaborative scenarios confronts fundamental challenges: privacy vulnerabilities, communication overhead, and computational bottlenecks.<n>We propose Federated Attention (FedAttn), which integrates the federated paradigm into the self-attention mechanism.
  arXiv  Detail & Related papers  (2025-11-04T15:14:58Z)
• MINERVA: Mutual Information Neural Estimation for Supervised Feature Selection [39.57737590420284]
  We introduce a novel approach to supervised feature selection based on neural estimation of mutual information between features and targets.<n>We paramaterize the approximation of mutual information with neural networks and perform feature selection using a carefully designed loss function augmented with sparsity-inducing regularizers.<n>Our method is implemented in a two-stage process to decouple representation learning from feature selection, ensuring better generalization and a more accurate expression of feature importance.
  arXiv  Detail & Related papers  (2025-10-02T23:09:06Z)
• Information-theoretic Quantification of High-order Feature Effects in Classification Problems [0.19791587637442676]
  We present an information-theoretic extension of the High-order interactions for Feature importance (Hi-Fi) method.<n>Our framework decomposes feature contributions into unique, synergistic, and redundant components.<n>Results indicate that the proposed estimator accurately recovers theoretical and expected findings.
  arXiv  Detail & Related papers  (2025-07-06T11:50:30Z)
• SocialMOIF: Multi-Order Intention Fusion for Pedestrian Trajectory Prediction [21.780343024406285]
  SocialMOIF is proposed to tackle these challenges, concentrating on the higher-order intention interactions among neighboring groups. Within SocialMOIF, a trajectory distribution approximator is designed to guide the trajectories toward values that align more closely with the actual data. A global trajectory is introduced to enable more accurate and efficient parallel predictions.
  arXiv  Detail & Related papers  (2025-04-22T06:14:49Z)
• Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis [55.13545823385091]
  Federated reinforcement learning (FedRL) enables collaborative learning while preserving data privacy by preventing direct data exchange between agents. In real-world applications, each agent may experience slightly different transition dynamics, leading to inherent model mismatches. We show that even moderate levels of information sharing can significantly mitigate environment-specific errors.
  arXiv  Detail & Related papers  (2025-03-21T18:06:28Z)
• Targeted Learning for Variable Importance [23.428985354228672]
  variable importance is one of the most widely used measures for interpreting machine learning. We propose a novel method by employing the targeted learning (TL) framework, designed to enhance robustness in inference for variable importance metrics. We show that it (i) retains the efficiency of traditional methods, (ii) maintains comparable computational complexity, and (iii) delivers improved accuracy, especially in finite sample contexts.
  arXiv  Detail & Related papers  (2024-11-04T16:14:45Z)
• Exploiting Structure in Offline Multi-Agent RL: The Benefits of Low Interaction Rank [52.831993899183416]
  We introduce a structural assumption -- the interaction rank -- and establish that functions with low interaction rank are significantly more robust to distribution shift compared to general ones. We demonstrate that utilizing function classes with low interaction rank, when combined with regularization and no-regret learning, admits decentralized, computationally and statistically efficient learning in offline MARL.
  arXiv  Detail & Related papers  (2024-10-01T22:16:22Z)
• Enhancing Interaction Modeling with Agent Selection and Physical Coefficient for Trajectory Prediction [1.6954753390775528]
  We present ASPILin, which manually selects interacting agents and calculates their correlations instead of attention scores. Remarkably, experiments conducted on the INTERACTION, highD, and CitySim datasets demonstrate that our method is efficient and straightforward.
  arXiv  Detail & Related papers  (2024-05-21T18:45:18Z)
• AntEval: Evaluation of Social Interaction Competencies in LLM-Driven Agents [65.16893197330589]
  Large Language Models (LLMs) have demonstrated their ability to replicate human behaviors across a wide range of scenarios. However, their capability in handling complex, multi-character social interactions has yet to be fully explored. We introduce the Multi-Agent Interaction Evaluation Framework (AntEval), encompassing a novel interaction framework and evaluation methods.
  arXiv  Detail & Related papers  (2024-01-12T11:18:00Z)
• IPCC-TP: Utilizing Incremental Pearson Correlation Coefficient for Joint Multi-Agent Trajectory Prediction [73.25645602768158]
  IPCC-TP is a novel relevance-aware module based on Incremental Pearson Correlation Coefficient to improve multi-agent interaction modeling. Our module can be conveniently embedded into existing multi-agent prediction methods to extend original motion distribution decoders.
  arXiv  Detail & Related papers  (2023-03-01T15:16:56Z)
• Variable Importance Matching for Causal Inference [73.25504313552516]
  We describe a general framework called Model-to-Match that achieves these goals. Model-to-Match uses variable importance measurements to construct a distance metric. We operationalize the Model-to-Match framework with LASSO.
  arXiv  Detail & Related papers  (2023-02-23T00:43:03Z)
• Inference for Interpretable Machine Learning: Fast, Model-Agnostic Confidence Intervals for Feature Importance [1.2891210250935146]
  We develop confidence intervals for a widely-used form of machine learning interpretation: feature importance. We do so by leveraging a form of random observation and feature subsampling called minipatch ensembles. Our approach is fast as computations needed for inference come nearly for free as part of the ensemble learning process.
  arXiv  Detail & Related papers  (2022-06-05T03:14:48Z)
• Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning [53.17258888552998]
  This work proposes an exploration variant of the basic $Q$-learning protocol with linear function approximation. We show that the performance of the algorithm degrades very gracefully under a novel and more permissive notion of approximation error.
  arXiv  Detail & Related papers  (2022-06-01T23:26:51Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• RR-Net: Injecting Interactive Semantics in Human-Object Interaction Detection [40.65483058890176]
  Latest end-to-end HOI detectors are short of relation reasoning, which leads to inability to learn HOI-specific interactive semantics for predictions. We first present a progressive Relation-aware Frame, which brings a new structure and parameter sharing pattern for interaction inference. Based on modules above, we construct an end-to-end trainable framework named Relation Reasoning Network (abbr. RR-Net)
  arXiv  Detail & Related papers  (2021-04-30T14:03:10Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)
• Robust Unsupervised Learning of Temporal Dynamic Interactions [21.928675010305543]
  In this paper we introduce a model-free metric based on the Procrustes distance for robust representation learning of interactions. We also introduce an optimal transport based distance metric for comparing between distributions of interaction primitives. Their usefulness will be demonstrated in unsupervised learning of vehicle-to-vechicle interactions extracted from the Safety Pilot database.
  arXiv  Detail & Related papers  (2020-06-18T02:39:45Z)
• Understanding Global Feature Contributions With Additive Importance Measures [14.50261153230204]
  We explore the perspective of defining feature importance through the predictive power associated with each feature. We introduce two notions of predictive power (model-based and universal) and formalize this approach with a framework of additive importance measures. We then propose SAGE, a model-agnostic method that quantifies predictive power while accounting for feature interactions.
  arXiv  Detail & Related papers  (2020-04-01T19:17:58Z)
• Cascaded Human-Object Interaction Recognition [175.60439054047043]
  We introduce a cascade architecture for a multi-stage, coarse-to-fine HOI understanding. At each stage, an instance localization network progressively refines HOI proposals and feeds them into an interaction recognition network. With our carefully-designed human-centric relation features, these two modules work collaboratively towards effective interaction understanding.
  arXiv  Detail & Related papers  (2020-03-09T17:05:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.