Evolutionary Feature Engineering for Structured Data

• URL: http://arxiv.org/abs/2607.01548v1
• Date: Thu, 02 Jul 2026 00:11:29 GMT
• Title: Evolutionary Feature Engineering for Structured Data
• Abstract summary: EFE represents transformations as Python programs with a standardized fit/transform interface.<n>For time-series forecasting, EFE-Time learns dataset-specific normalizations that improve off-the-shelf time-series foundation models.<n>EFE-Tab evolves compact feature programs that add useful interpretable features and remove redundant ones.
• Score: 26.2268110175559
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large language models are increasingly used as open-ended search operators in evolutionary optimization. We introduce Evolutionary Feature Engineering (EFE), a framework for using LLM-based evolution to discover preprocessing transformations for structured data. EFE represents transformations as Python programs with a standardized fit/transform interface, allowing them to be inserted directly into existing machine learning pipelines. During evolution, candidate programs are refined using dataset context, summary statistics, and downstream performance feedback on validation set. We instantiate EFE in two settings. For time-series forecasting, EFE-Time learns invertible, dataset-specific normalizations that improve off-the-shelf time-series foundation models. It reduces forecasting errors (MASE, WQL, MAE) 3% or more when averaged across datasets and improvements are as much as 19% on the COVID-Deaths dataset. Notably, these improvements occur with recent TSFMs such as Chronos-2. For tabular prediction, EFE-Tab evolves compact feature programs that add useful interpretable features and remove redundant ones, improving or matching existing LLM-based feature-engineering methods. We found EFE-Tab to be particularly effective on classical decision trees, where small sets of evolved features yield competitive accuracy while preserving interpretability. Overall, EFE demonstrates that LLM-based evolution can improve both accuracy and interpretability when automatically tackling structured data.

Related papers

• TopoFE: topology-aware LLM-guided Automated Feature Engineering [22.323342919724492]
  TOPOFE is a topology-aware evolutionary framework for LLM-guided feature engineering.<n>We show consistent improvements over state-of-the-art AutoFE methods across classification and regression tasks.
  arXiv  Detail & Related papers  (2026-07-25T16:45:00Z)
• DataEvolver: Automatic Data Preparation for Large Language Models through Multi-Level Self-Evolving [55.85201665674226]
  High-quality training data is essential to large language models (LLMs)<n>Existing automatic data preparation methods rely on predefined pipelines or customized human instructions.<n>We introduce DataEvolver, the first self-evolving data preparation system that automatically constructs pipelines to transform raw data into high-quality data.
  arXiv  Detail & Related papers  (2026-06-05T07:44:34Z)
• EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptation [79.71802168256542]
  Adapting large language models (LLMs) to a targeted task efficiently and effectively remains a fundamental challenge.<n>One straightforward approach is through an iterative generation-training loop, where candidate data are synthesized through an external generator.<n>We introduce a refined paradigm, namely an iterative generation-selection-training loop, which incorporates a selection step prior to model updates.
  arXiv  Detail & Related papers  (2026-04-28T23:26:16Z)
• From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models [73.72877445629383]
  Interpretability-Guided Data Selection (IGDS) is a framework that first identifies causal task features through frequency recall and interventional filtering.<n>We validate IGDS on mathematical reasoning, summarization, and translation tasks within Gemma-2, LLaMA-3.1, and Qwen3 models.
  arXiv  Detail & Related papers  (2026-04-28T03:16:24Z)
• Learning from the Undesirable: Robust Adaptation of Language Models without Forgetting [18.680059467974825]
  Language models (LMs) are often adapted through supervised fine-tuning (SFT) to specialize their capabilities for downstream tasks.<n>In typical scenarios where the fine-tuning data is limited, SFT can lead LMs to overfit, causing them to rely on spurious patterns.<n>We propose Learning-from-the-Undesirable (LfU), a simple yet effective regularization scheme for SFT to mitigate issues when fine-tuning LMs with limited data.
  arXiv  Detail & Related papers  (2025-11-17T06:57:44Z)
• Estimating Time Series Foundation Model Transferability via In-Context Learning [74.65355820906355]
  Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training.<n>Fine-tuning remains critical for boosting performance in domains with limited public data.<n>We introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem.
  arXiv  Detail & Related papers  (2025-09-28T07:07:13Z)
• LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers [10.282327560070202]
  Large Language Models (LLMs) have enabled the integration of domain knowledge into the feature engineering process.<n>We propose LLM-FE, a novel framework that combines evolutionary search with the domain knowledge and reasoning capabilities of LLMs.<n>Our results demonstrate that LLM-FE consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-03-18T17:11:24Z)
• Meta-Statistical Learning: Supervised Learning of Statistical Inference [59.463430294611626]
  This work demonstrates that the tools and principles driving the success of large language models (LLMs) can be repurposed to tackle distribution-level tasks.<n>We propose meta-statistical learning, a framework inspired by multi-instance learning that reformulates statistical inference tasks as supervised learning problems.
  arXiv  Detail & Related papers  (2025-02-17T18:04:39Z)
• APAR: Modeling Irregular Target Functions in Tabular Regression via Arithmetic-Aware Pre-Training and Adaptive-Regularized Fine-Tuning [12.35924469567586]
  We propose a novel Arithmetic-Aware Pre-training and Adaptive-Regularized Fine-tuning framework (APAR)<n>In the pre-training phase, APAR introduces an arithmetic-aware pretext objective to capture intricate sample-wise relationships from the perspective of continuous labels.<n>In the fine-tuning phase, a consistency-based adaptive regularization technique is proposed to self-learn appropriate data augmentation.
  arXiv  Detail & Related papers  (2024-12-14T19:33:21Z)
• Structural Entropy Guided Probabilistic Coding [52.01765333755793]
  We propose a novel structural entropy-guided probabilistic coding model, named SEPC.<n>We incorporate the relationship between latent variables into the optimization by proposing a structural entropy regularization loss.<n> Experimental results across 12 natural language understanding tasks, including both classification and regression tasks, demonstrate the superior performance of SEPC.
  arXiv  Detail & Related papers  (2024-12-12T00:37:53Z)
• A Transformer-based Framework For Multi-variate Time Series: A Remaining Useful Life Prediction Use Case [4.0466311968093365]
  This work proposed an encoder-transformer architecture-based framework for time series prediction. We validated the effectiveness of the proposed framework on all four sets of the C-MAPPS benchmark dataset. To enable the model awareness of the initial stages of the machine life and its degradation path, a novel expanding window method was proposed.
  arXiv  Detail & Related papers  (2023-08-19T02:30:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.