Learning High-Order Interactions via Targeted Pattern Search
- URL: http://arxiv.org/abs/2102.12974v1
- Date: Tue, 23 Feb 2021 11:13:22 GMT
- Title: Learning High-Order Interactions via Targeted Pattern Search
- Authors: Michela C. Massi, Nicola R. Franco, Francesca Ieva, Andrea Manzoni,
Anna Maria Paganoni, Paolo Zunino
- Abstract summary: We present a novel algorithm, Learning high-order Interactions via targeted Pattern Search (LIPS)
LIPS selects interaction terms of varying order to include in a Logistic Regression model.
We prove its wide applicability to real-life research scenarios, showing that it outperforms a benchmark state-of-the-art algorithm.
- Score: 0.6198237241838558
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Logistic Regression (LR) is a widely used statistical method in empirical
binary classification studies. However, real-life scenarios oftentimes share
complexities that prevent from the use of the as-is LR model, and instead
highlight the need to include high-order interactions to capture data
variability. This becomes even more challenging because of: (i) datasets
growing wider, with more and more variables; (ii) studies being typically
conducted in strongly imbalanced settings; (iii) samples going from very large
to extremely small; (iv) the need of providing both predictive models and
interpretable results. In this paper we present a novel algorithm, Learning
high-order Interactions via targeted Pattern Search (LIPS), to select
interaction terms of varying order to include in a LR model for an imbalanced
binary classification task when input data are categorical. LIPS's rationale
stems from the duality between item sets and categorical interactions. The
algorithm relies on an interaction learning step based on a well-known frequent
item set mining algorithm, and a novel dissimilarity-based interaction
selection step that allows the user to specify the number of interactions to be
included in the LR model. In addition, we particularize two variants (Scores
LIPS and Clusters LIPS), that can address even more specific needs. Through a
set of experiments we validate our algorithm and prove its wide applicability
to real-life research scenarios, showing that it outperforms a benchmark
state-of-the-art algorithm.
Related papers
- Binary Code Similarity Detection via Graph Contrastive Learning on Intermediate Representations [52.34030226129628]
Binary Code Similarity Detection (BCSD) plays a crucial role in numerous fields, including vulnerability detection, malware analysis, and code reuse identification.
In this paper, we propose IRBinDiff, which mitigates compilation differences by leveraging LLVM-IR with higher-level semantic abstraction.
Our extensive experiments, conducted under varied compilation settings, demonstrate that IRBinDiff outperforms other leading BCSD methods in both One-to-one comparison and One-to-many search scenarios.
arXiv Detail & Related papers (2024-10-24T09:09:20Z) - Interpetable Target-Feature Aggregation for Multi-Task Learning based on Bias-Variance Analysis [53.38518232934096]
Multi-task learning (MTL) is a powerful machine learning paradigm designed to leverage shared knowledge across tasks to improve generalization and performance.
We propose an MTL approach at the intersection between task clustering and feature transformation based on a two-phase iterative aggregation of targets and features.
In both phases, a key aspect is to preserve the interpretability of the reduced targets and features through the aggregation with the mean, which is motivated by applications to Earth science.
arXiv Detail & Related papers (2024-06-12T08:30:16Z) - Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity [59.57065228857247]
Retrieval-augmented Large Language Models (LLMs) have emerged as a promising approach to enhancing response accuracy in several tasks, such as Question-Answering (QA)
We propose a novel adaptive QA framework, that can dynamically select the most suitable strategy for (retrieval-augmented) LLMs based on the query complexity.
We validate our model on a set of open-domain QA datasets, covering multiple query complexities, and show that ours enhances the overall efficiency and accuracy of QA systems.
arXiv Detail & Related papers (2024-03-21T13:52:30Z) - Collaborative Learning with Different Labeling Functions [7.228285747845779]
We study a variant of Collaborative PAC Learning, in which we aim to learn an accurate classifier for each of the $n$ data distributions.
We show that, when the data distributions satisfy a weaker realizability assumption, sample-efficient learning is still feasible.
arXiv Detail & Related papers (2024-02-16T04:32:22Z) - Variable Importance in High-Dimensional Settings Requires Grouping [19.095605415846187]
Conditional Permutation Importance (CPI) bypasses PI's limitations in such cases.
Grouping variables statistically via clustering or some prior knowledge gains some power back.
We show that the approach extended with stacking controls the type-I error even with highly-correlated groups.
arXiv Detail & Related papers (2023-12-18T00:21:47Z) - RGM: A Robust Generalizable Matching Model [49.60975442871967]
We propose a deep model for sparse and dense matching, termed RGM (Robust Generalist Matching)
To narrow the gap between synthetic training samples and real-world scenarios, we build a new, large-scale dataset with sparse correspondence ground truth.
We are able to mix up various dense and sparse matching datasets, significantly improving the training diversity.
arXiv Detail & Related papers (2023-10-18T07:30:08Z) - Improved Robust Algorithms for Learning with Discriminative Feature
Feedback [21.58493386054356]
Discriminative Feature Feedback is a protocol for interactive learning based on feature explanations that are provided by a human teacher.
We provide new robust interactive learning algorithms for the Discriminative Feature Feedback model.
arXiv Detail & Related papers (2022-09-08T12:11:12Z) - A Lagrangian Duality Approach to Active Learning [119.36233726867992]
We consider the batch active learning problem, where only a subset of the training data is labeled.
We formulate the learning problem using constrained optimization, where each constraint bounds the performance of the model on labeled samples.
We show, via numerical experiments, that our proposed approach performs similarly to or better than state-of-the-art active learning methods.
arXiv Detail & Related papers (2022-02-08T19:18:49Z) - pRSL: Interpretable Multi-label Stacking by Learning Probabilistic Rules [0.0]
We present the probabilistic rule stacking (pRSL) which uses probabilistic propositional logic rules and belief propagation to combine the predictions of several underlying classifiers.
We derive algorithms for exact and approximate inference and learning, and show that pRSL reaches state-of-the-art performance on various benchmark datasets.
arXiv Detail & Related papers (2021-05-28T14:06:21Z) - Few-Shot Named Entity Recognition: A Comprehensive Study [92.40991050806544]
We investigate three schemes to improve the model generalization ability for few-shot settings.
We perform empirical comparisons on 10 public NER datasets with various proportions of labeled data.
We create new state-of-the-art results on both few-shot and training-free settings.
arXiv Detail & Related papers (2020-12-29T23:43:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.