Feature Selection Methods for Uplift Modeling and Heterogeneous Treatment Effect

• URL: http://arxiv.org/abs/2005.03447v2
• Date: Fri, 8 Jul 2022 17:46:21 GMT
• Title: Feature Selection Methods for Uplift Modeling and Heterogeneous Treatment Effect
• Authors: Zhenyu Zhao, Yumin Zhang, Totte Harinen, Mike Yung
• Abstract summary: Uplift modeling is a causal learning technique that estimates subgroup-level treatment effects. Traditional methods for doing feature selection are not fit for the task. We introduce a set of feature selection methods explicitly designed for uplift modeling.
• Score: 1.349645012479288
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Uplift modeling is a causal learning technique that estimates subgroup-level treatment effects. It is commonly used in industry and elsewhere for tasks such as targeting ads. In a typical setting, uplift models can take thousands of features as inputs, which is costly and results in problems such as overfitting and poor model interpretability. Consequently, there is a need to select a subset of the most important features for modeling. However, traditional methods for doing feature selection are not fit for the task because they are designed for standard machine learning models whose target is importantly different from uplift models. To address this, we introduce a set of feature selection methods explicitly designed for uplift modeling, drawing inspiration from statistics and information theory. We conduct empirical evaluations on the proposed methods on publicly available datasets, demonstrating the advantages of the proposed methods compared to traditional feature selection. We make the proposed methods publicly available as a part of the CausalML open-source package.

Related papers

• Influence Functions for Scalable Data Attribution in Diffusion Models [52.92223039302037]
  Diffusion models have led to significant advancements in generative modelling. Yet their widespread adoption poses challenges regarding data attribution and interpretability. In this paper, we aim to help address such challenges by developing an textitinfluence functions framework.
  arXiv  Detail & Related papers  (2024-10-17T17:59:02Z)
• Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning [78.72226641279863]
  Sparse Mixture of Expert (SMoE) models have emerged as a scalable alternative to dense models in language modeling. Our research explores task-specific model pruning to inform decisions about designing SMoE architectures. We introduce an adaptive task-aware pruning technique UNCURL to reduce the number of experts per MoE layer in an offline manner post-training.
  arXiv  Detail & Related papers  (2024-09-02T22:35:03Z)
• Binary Feature Mask Optimization for Feature Selection [0.0]
  We introduce a novel framework that selects features considering the predictions of the model. Our framework innovates by using a novel feature masking approach to eliminate the features during the selection process. We demonstrate significant performance improvements on the real-life datasets using LightGBM and Multi-Layer Perceptron as our ML models.
  arXiv  Detail & Related papers  (2024-01-23T10:54:13Z)
• Extending Variability-Aware Model Selection with Bias Detection in Machine Learning Projects [0.7646713951724013]
  This paper describes work on extending an adaptive variability-aware model selection method with bias detection in machine learning projects. The proposed approach aims to advance the state of the art by making explicit factors that influence model selection, particularly those related to bias, as well as their interactions.
  arXiv  Detail & Related papers  (2023-11-23T22:08:29Z)
• Towards Efficient Task-Driven Model Reprogramming with Foundation Models [52.411508216448716]
  Vision foundation models exhibit impressive power, benefiting from the extremely large model capacity and broad training data. However, in practice, downstream scenarios may only support a small model due to the limited computational resources or efficiency considerations. This brings a critical challenge for the real-world application of foundation models: one has to transfer the knowledge of a foundation model to the downstream task.
  arXiv  Detail & Related papers  (2023-04-05T07:28:33Z)
• PAMI: partition input and aggregate outputs for model interpretation [69.42924964776766]
  In this study, a simple yet effective visualization framework called PAMI is proposed based on the observation that deep learning models often aggregate features from local regions for model predictions. The basic idea is to mask majority of the input and use the corresponding model output as the relative contribution of the preserved input part to the original model prediction. Extensive experiments on multiple tasks confirm the proposed method performs better than existing visualization approaches in more precisely finding class-specific input regions.
  arXiv  Detail & Related papers  (2023-02-07T08:48:34Z)
• Multidimensional Item Response Theory in the Style of Collaborative Filtering [0.8057006406834467]
  This paper presents a machine learning approach to multidimensional item response theory (MIRT) Inspired by collaborative filtering, we define a general class of models that includes many MIRT models. We discuss the use of penalized joint maximum likelihood (JML) to estimate individual models and cross-validation to select the best performing model.
  arXiv  Detail & Related papers  (2023-01-03T00:56:27Z)
• Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers [66.36045164286854]
  We analyze a set of existing bias features and demonstrate there is no single model that works best for all the cases. By choosing an appropriate bias model, we can obtain a better robustness result than baselines with a more sophisticated model design.
  arXiv  Detail & Related papers  (2022-10-28T17:52:10Z)
• Deep Learning for Choice Modeling [5.173001988341294]
  We develop deep learning-based choice models under two settings of choice modeling: feature-free and feature-based. Our model captures both the intrinsic utility for each candidate choice and the effect that the assortment has on the choice probability.
  arXiv  Detail & Related papers  (2022-08-19T13:10:17Z)
• An Additive Instance-Wise Approach to Multi-class Model Interpretation [53.87578024052922]
  Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. Existing methods mainly focus on selecting explanatory input features, which follow either locally additive or instance-wise approaches. This work exploits the strengths of both methods and proposes a global framework for learning local explanations simultaneously for multiple target classes.
  arXiv  Detail & Related papers  (2022-07-07T06:50:27Z)
• Model-specific Data Subsampling with Influence Functions [37.64859614131316]
  We develop a model-specific data subsampling strategy that improves over random sampling whenever training points have varying influence. Specifically, we leverage influence functions to guide our selection strategy, proving theoretically, and demonstrating empirically that our approach quickly selects high-quality models.
  arXiv  Detail & Related papers  (2020-10-20T12:10:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.