Feature Interaction Aware Automated Data Representation Transformation

• URL: http://arxiv.org/abs/2309.17011v2
• Date: Mon, 15 Jan 2024 09:02:16 GMT
• Title: Feature Interaction Aware Automated Data Representation Transformation
• Authors: Ehtesamul Azim, Dongjie Wang, Kunpeng Liu, Wei Zhang, Yanjie Fu
• Abstract summary: We develop a hierarchical reinforcement learning structure with cascading Markov Decision Processes to automate feature and operation selection. We reward agents based on the interaction strength between selected features, resulting in intelligent and efficient exploration of the feature space that emulates human decision-making.
• Score: 27.26916497306978
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Creating an effective representation space is crucial for mitigating the curse of dimensionality, enhancing model generalization, addressing data sparsity, and leveraging classical models more effectively. Recent advancements in automated feature engineering (AutoFE) have made significant progress in addressing various challenges associated with representation learning, issues such as heavy reliance on intensive labor and empirical experiences, lack of explainable explicitness, and inflexible feature space reconstruction embedded into downstream tasks. However, these approaches are constrained by: 1) generation of potentially unintelligible and illogical reconstructed feature spaces, stemming from the neglect of expert-level cognitive processes; 2) lack of systematic exploration, which subsequently results in slower model convergence for identification of optimal feature space. To address these, we introduce an interaction-aware reinforced generation perspective. We redefine feature space reconstruction as a nested process of creating meaningful features and controlling feature set size through selection. We develop a hierarchical reinforcement learning structure with cascading Markov Decision Processes to automate feature and operation selection, as well as feature crossing. By incorporating statistical measures, we reward agents based on the interaction strength between selected features, resulting in intelligent and efficient exploration of the feature space that emulates human decision-making. Extensive experiments are conducted to validate our proposed approach.

Related papers

• Dynamic and Adaptive Feature Generation with LLM [10.142660254703225]
  We propose a dynamic and adaptive feature generation method that enhances the interpretability of the feature generation process. Our approach broadens the applicability across various data types and tasks and draws advantages over strategic flexibility.
  arXiv  Detail & Related papers  (2024-06-04T20:32:14Z)
• Traceable Group-Wise Self-Optimizing Feature Transformation Learning: A Dual Optimization Perspective [33.45878576396101]
  Feature transformation aims to reconstruct an effective representation space by mathematically refining the existing features. Existing research predominantly focuses on domain knowledge-based feature engineering or learning latent representations. Our initial work took a pioneering step towards this challenge by introducing a novel self-optimizing framework.
  arXiv  Detail & Related papers  (2023-06-29T12:29:21Z)
• Traceable Automatic Feature Transformation via Cascading Actor-Critic Agents [25.139229855367088]
  Feature transformation is an essential task to boost the effectiveness and interpretability of machine learning (ML) We formulate the feature transformation task as an iterative, nested process of feature generation and selection. We show 24.7% improvements in F1 scores compared with SOTAs and robustness in high-dimensional data.
  arXiv  Detail & Related papers  (2022-12-27T08:20:19Z)
• Self-Optimizing Feature Transformation [33.458785763961004]
  Feature transformation aims to extract a good representation (feature) space by mathematically transforming existing features. Current research focuses on domain knowledge-based feature engineering or learning latent representations. We present a self-optimizing framework for feature transformation.
  arXiv  Detail & Related papers  (2022-09-16T16:50:41Z)
• Group-wise Reinforcement Feature Generation for Optimal and Explainable Representation Space Reconstruction [25.604176830832586]
  We reformulate representation space reconstruction into an interactive process of nested feature generation and selection. We design a group-wise generation strategy to cross a feature group, an operation, and another feature group to generate new features. We present extensive experiments to demonstrate the effectiveness, efficiency, traceability, and explicitness of our system.
  arXiv  Detail & Related papers  (2022-05-28T21:34:14Z)
• Pessimism meets VCG: Learning Dynamic Mechanism Design via Offline Reinforcement Learning [114.36124979578896]
  We design a dynamic mechanism using offline reinforcement learning algorithms. Our algorithm is based on the pessimism principle and only requires a mild assumption on the coverage of the offline data set.
  arXiv  Detail & Related papers  (2022-05-05T05:44:26Z)
• Information-Theoretic Odometry Learning [83.36195426897768]
  We propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation. The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language.
  arXiv  Detail & Related papers  (2022-03-11T02:37:35Z)
• Online reinforcement learning with sparse rewards through an active inference capsule [62.997667081978825]
  This paper introduces an active inference agent which minimizes the novel free energy of the expected future. Our model is capable of solving sparse-reward problems with a very high sample efficiency. We also introduce a novel method for approximating the prior model from the reward function, which simplifies the expression of complex objectives.
  arXiv  Detail & Related papers  (2021-06-04T10:03:36Z)
• Progressive Self-Guided Loss for Salient Object Detection [102.35488902433896]
  We present a progressive self-guided loss function to facilitate deep learning-based salient object detection in images. Our framework takes advantage of adaptively aggregated multi-scale features to locate and detect salient objects effectively.
  arXiv  Detail & Related papers  (2021-01-07T07:33:38Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)
• A Dependency Syntactic Knowledge Augmented Interactive Architecture for End-to-End Aspect-based Sentiment Analysis [73.74885246830611]
  We propose a novel dependency syntactic knowledge augmented interactive architecture with multi-task learning for end-to-end ABSA. This model is capable of fully exploiting the syntactic knowledge (dependency relations and types) by leveraging a well-designed Dependency Relation Embedded Graph Convolutional Network (DreGcn) Extensive experimental results on three benchmark datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-04T14:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.