Scalable variable selection for two-view learning tasks with projection operators

• URL: http://arxiv.org/abs/2307.01558v1
• Date: Tue, 4 Jul 2023 08:22:05 GMT
• Title: Scalable variable selection for two-view learning tasks with projection operators
• Authors: Sandor Szedmak (1), Riikka Huusari (1), Tat Hong Duong Le (1), Juho Rousu (1) ((1) Department of Computer Science, Aalto University, Espoo, Finland)
• Abstract summary: We propose a novel variable selection method for two-view settings, or for vector-valued supervised learning problems. Our framework is able to handle extremely large scale selection tasks, where number of data samples could be even millions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we propose a novel variable selection method for two-view settings, or for vector-valued supervised learning problems. Our framework is able to handle extremely large scale selection tasks, where number of data samples could be even millions. In a nutshell, our method performs variable selection by iteratively selecting variables that are highly correlated with the output variables, but which are not correlated with the previously chosen variables. To measure the correlation, our method uses the concept of projection operators and their algebra. With the projection operators the relationship, correlation, between sets of input and output variables can also be expressed by kernel functions, thus nonlinear correlation models can be exploited as well. We experimentally validate our approach, showing on both synthetic and real data its scalability and the relevance of the selected features. Keywords: Supervised variable selection, vector-valued learning, projection-valued measure, reproducing kernel Hilbert space

Related papers

• Knowledge Composition using Task Vectors with Learned Anisotropic Scaling [51.4661186662329]
  We introduce aTLAS, an algorithm that linearly combines parameter blocks with different learned coefficients, resulting in anisotropic scaling at the task vector level. We show that such linear combinations explicitly exploit the low intrinsicity of pre-trained models, with only a few coefficients being the learnable parameters. We demonstrate the effectiveness of our method in task arithmetic, few-shot recognition and test-time adaptation, with supervised or unsupervised objectives.
  arXiv  Detail & Related papers  (2024-07-03T07:54:08Z)
• Contextual Feature Selection with Conditional Stochastic Gates [9.784482648233048]
  Conditional Gates (c-STG) models the importance of features using conditional variables whose parameters are predicted based on contextual variables. We show that c-STG can lead to improved feature selection capabilities while enhancing prediction accuracy and interpretability.
  arXiv  Detail & Related papers  (2023-12-21T19:12:59Z)
• Data-driven path collective variables [0.0]
  We propose a new method for the generation, optimization, and comparison of collective variables. The resulting collective variable is one-dimensional, interpretable, and differentiable. We demonstrate the validity of the method on two different applications.
  arXiv  Detail & Related papers  (2023-12-21T14:07:47Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• DiscoVars: A New Data Analysis Perspective -- Application in Variable Selection for Clustering [0.0]
  We present a new data analysis perspective to determine variable importance regardless of the underlying learning task. We propose a new methodology to select important variables from the data by first creating dependency networks among all variables. We present our tool as a Shiny app which is a user-friendly interface development environment.
  arXiv  Detail & Related papers  (2023-04-08T10:57:19Z)
• A Lagrangian Duality Approach to Active Learning [119.36233726867992]
  We consider the batch active learning problem, where only a subset of the training data is labeled. We formulate the learning problem using constrained optimization, where each constraint bounds the performance of the model on labeled samples. We show, via numerical experiments, that our proposed approach performs similarly to or better than state-of-the-art active learning methods.
  arXiv  Detail & Related papers  (2022-02-08T19:18:49Z)
• VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning [84.70916463298109]
  VarCLR is a new approach for learning semantic representations of variable names. VarCLR is an excellent fit for contrastive learning, which aims to minimize the distance between explicitly similar inputs. We show that VarCLR enables the effective application of sophisticated, general-purpose language models like BERT.
  arXiv  Detail & Related papers  (2021-12-05T18:40:32Z)
• A Two-Stage Variable Selection Approach for Correlated High Dimensional Predictors [4.8128078741263725]
  We propose a two-stage approach that combines a variable clustering stage and a group variable stage for the group variable selection problem. The variable clustering stage uses information from the data to find a group structure, which improves the performance of the existing group variable selection methods. The two-stage method shows a better performance, in terms of the prediction accuracy, as well as in the accuracy to select active predictors.
  arXiv  Detail & Related papers  (2021-03-24T17:28:34Z)
• Visual Neural Decomposition to Explain Multivariate Data Sets [13.117139248511783]
  Investigating relationships between variables in multi-dimensional data sets is a common task for data analysts and engineers. We propose a novel approach to visualize correlations between input variables and a target output variable that scales to hundreds of variables.
  arXiv  Detail & Related papers  (2020-09-11T15:53:37Z)
• Improving Multi-Turn Response Selection Models with Complementary Last-Utterance Selection by Instance Weighting [84.9716460244444]
  We consider utilizing the underlying correlation in the data resource itself to derive different kinds of supervision signals. We conduct extensive experiments in two public datasets and obtain significant improvement in both datasets.
  arXiv  Detail & Related papers  (2020-02-18T06:29:01Z)
• Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages [112.65994041398481]
  We propose a Bayesian generative model for the space of neural parameters. We infer the posteriors over such latent variables based on data from seen task-language combinations. Our model yields comparable or better results than state-of-the-art, zero-shot cross-lingual transfer methods.
  arXiv  Detail & Related papers  (2020-01-30T16:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.