LATTEArena: An Evaluation Framework for LLM-powered Tabular Feature Engineering (Extended Version)

• URL: http://arxiv.org/abs/2606.09004v2
• Date: Tue, 16 Jun 2026 11:38:16 GMT
• Title: LATTEArena: An Evaluation Framework for LLM-powered Tabular Feature Engineering (Extended Version)
• Abstract summary: We introduce LATTEArena, a standardized, modular, and benchmarking framework for automated feature engineering.<n>By enabling controlled component-level comparisons, LATTEArena shifts the paradigm from ad-hoc prompt engineering to systematic context management.<n>All code, datasets, and over 4,000 execution logs are publicly available to foster a dynamic, community-driven benchmark.
• Score: 21.873402103599588
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Feature engineering remains a cornerstone of tabular data analysis, and Large Language Models (LLMs) have emerged as a promising paradigm for its automation, giving rise to LLM-powered Automated Tabular Feature Engineering (LATTE). However, the field lacks standardized, cost-aware evaluation platforms, and the combinatorial explosion of design choices obscures true algorithmic progress. To bridge these gaps, we systematically deconstruct 15 representative LATTE methods into a unified 6-dimensional taxonomy. Based on this abstraction, we introduce LATTEArena, a standardized, modular, and extensible benchmarking framework that decouples monolithic pipelines into reusable execution blocks. By distilling the massive combinatorial space, we evaluate 24 core LATTE configurations across 7 research questions. Our head-to-head benchmarking goes beyond predictive accuracy to quantify token efficiency and execution robustness, yielding 17 empirical findings on cost-effectiveness trade-offs. Furthermore, we provide 3 concrete recommendations for optimal real-world deployment. By enabling controlled component-level comparisons, LATTEArena shifts the paradigm from ad-hoc prompt engineering to systematic context management. All code, datasets, and over 4,000 execution logs are publicly available to foster a dynamic, community-driven benchmark. Our framework, leaderboard, and all artifacts are hosted on the LATTEArena project website at https://goodenhak.github.io/LATTEArena.

Related papers

• Artificial Intelligence for Energy Optimization in Data Centers [0.0]
  Data centers are increasingly optimized by artificial intelligence and, at the same time, increasingly loaded by it.<n>We screen roughly 194 papers retrieved through a documented protocol, code 63 of them, and report what the coding shows.<n>We set out CLEARDC, a framework coupling a control-policy branch to a workload-demand branch through an explicit elasticity term, reads out net rather than direct benefit, and emits a schema-conformant record covering energy, carbon, water, embodied share and validation venue.
  arXiv  Detail & Related papers  (2026-09-03T11:51:15Z)
• OpenCompass: A Universal Evaluation Platform for Large Language Models [62.59671563145442]
  General-purpose large language models (LLMs) have become a critical link in advancing technological development.<n>Mainstream benchmark datasets face challenges such as the diversity of task types, inconsistent evaluation criteria, and fragmentation of data and processing.<n>This paper proposes and open-source Open LLMs, a one-stop, scalable, and high-concurrency general-purpose evaluation platform.
  arXiv  Detail & Related papers  (2026-05-19T02:50:11Z)
• Context, Reasoning, and Hierarchy: A Cost-Performance Study of Compound LLM Agent Design in an Adversarial POMDP [3.774094352572544]
  Programmatic state abstraction delivers the largest returns per token spent.<n>Distributing deliberation tools across a hierarchy degrades performance relative to hierarchy alone.<n>Hierarchical decomposition without deliberation achieves the best absolute performance for most models.
  arXiv  Detail & Related papers  (2026-05-15T17:23:08Z)
• BatchBench: Toward a Workload-Aware Benchmark for Autoscaling Policies in Big Data Batch Processing -- A Proposed Framework [0.0]
  BatchBench is an open benchmarking framework designed to place rule-based, learned, and agentic autoscaling policies on equal footing.<n>We contribute: (1) a workload of six batch processing classes synthesized from published autoscaling benchmarks and publicly released cluster traces; (2) a validation methodology based on two-sample Kolmogorov-Smirnov and earth-mover distance; (3) a five-axis evaluation harness specification covering cost, SLA attainment, scaling responsiveness, scaling thrash, and decision interpretability; and (4) a standardized agent interface that lets LLM-based and reinforcement-learning autoscalers be evaluated alongside rule-based controllers with
  arXiv  Detail & Related papers  (2026-05-12T15:36:20Z)
• MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI [92.82821272460423]
  MLS-Bench is a benchmark for evaluating whether AI systems can invent generalizable and scalable ML methods.<n>We study the effects of test-time scaling, adaptive compute allocation, and context provision on agents' discovery performance.
  arXiv  Detail & Related papers  (2026-05-09T04:29:46Z)
• FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios [58.34124792457706]
  The manufacturing sector is increasingly adopting Multimodal Large Language Models (MLLMs) to transition from simple perception to autonomous execution.<n>Progress is hindered by data scarcity and a lack of fine-grained domain semantics in existing datasets.<n>We first construct a high-quality dataset that combines realworld 2D images and 3D point clouds, annotated with fine-grained domain semantics.<n>We then evaluate 18 state-of-the-art MLLMs across three manufacturing tasks, namely verification, structural surface inspection, and assembly verification, revealing significant performance gaps.
  arXiv  Detail & Related papers  (2026-04-08T12:23:27Z)
• PymooLab: An Open-Source Visual Analytics Framework for Multi-Objective Optimization using LLM-Based Code Generation and MCDM [0.0]
  PymooLab is an open-source visual analytics environment built on top of textitpymoo.<n>It unifies configuration, execution monitoring, and formal decision support in a single reproducible workflow.<n>For computationally intensive studies, PymooLab relies on the native textitpymoo acceleration pathway through JAX.
  arXiv  Detail & Related papers  (2026-03-02T00:56:32Z)
• RAVEL: Reasoning Agents for Validating and Evaluating LLM Text Synthesis [78.32151470154422]
  We introduce RAVEL, an agentic framework that enables the testers to autonomously plan and execute typical synthesis operations.<n>We present C3EBench, a benchmark comprising 1,258 samples derived from professional human writings.<n>By augmenting RAVEL with SOTA LLMs as operators, we find that such agentic text synthesis is dominated by the LLM's reasoning capability.
  arXiv  Detail & Related papers  (2026-02-28T14:47:34Z)
• LogicLens: Visual-Logical Co-Reasoning for Text-Centric Forgery Analysis [10.305807834419765]
  Text-centric forgeries pose a significant threat to societal security and information authenticity.<n>Current methods for text-centric forgery analysis are often limited to coarse-grained visual analysis.<n>We introduce LogicLens, a unified framework for Visual-Textual Co-reasoning.
  arXiv  Detail & Related papers  (2025-12-25T03:02:27Z)
• Experts are all you need: A Composable Framework for Large Language Model Inference [8.747592414164687]
  Large Language Models (LLMs) have achieved state-of-the-art accuracies in a variety of natural language processing (NLP) tasks.<n>MoEs overcome this bottleneck by decoupling model capacity from computation by only activating a subset of parameters or "experts"
  arXiv  Detail & Related papers  (2025-11-28T08:00:16Z)
• Structured Prompting Enables More Robust Evaluation of Language Models [38.53918044830268]
  We present a DSPy+HELM framework that introduces structured prompting methods which elicit reasoning.<n>We find that without structured prompting, HELM underestimates LM performance (by 4% average) and performance estimates vary more across benchmarks.<n>This is the first benchmarking study to systematically integrate structured prompting into an established evaluation framework.
  arXiv  Detail & Related papers  (2025-11-25T20:37:59Z)
• What Matters in LLM-Based Feature Extractor for Recommender? A Systematic Analysis of Prompts, Models, and Adaptation [14.788780469735242]
  We propose RecXplore, a modular framework that decomposes the LLM-as-feature-extractor pipeline into four modules.<n>Instead of proposing new techniques, RecXplore revisits and organizes established methods, enabling systematic exploration of each module in isolation.<n>Experiments show that simply combining the best designs from existing techniques without exhaustive search yields up to 18.7% relative improvement in NDCG@5 and 12.7% in HR@5 over strong baselines.
  arXiv  Detail & Related papers  (2025-09-18T14:08:45Z)
• LLM4CMO: Large Language Model-aided Algorithm Design for Constrained Multiobjective Optimization [54.35609820607923]
  Large language models (LLMs) offer new opportunities for assisting with algorithm design.<n>We propose LLM4CMO, a novel CMOEA based on a dual-population, two-stage framework.<n>LLMs can serve as efficient co-designers in the development of complex evolutionary optimization algorithms.
  arXiv  Detail & Related papers  (2025-08-16T02:00:57Z)
• ZeroLM: Data-Free Transformer Architecture Search for Language Models [54.83882149157548]
  Current automated proxy discovery approaches suffer from extended search times, susceptibility to data overfitting, and structural complexity.<n>This paper introduces a novel zero-cost proxy methodology that quantifies model capacity through efficient weight statistics.<n>Our evaluation demonstrates the superiority of this approach, achieving a Spearman's rho of 0.76 and Kendall's tau of 0.53 on the FlexiBERT benchmark.
  arXiv  Detail & Related papers  (2025-03-24T13:11:22Z)
• LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning [56.273799410256075]
  The framework combines Monte Carlo Tree Search (MCTS) with iterative Self-Refine to optimize the reasoning path. The framework has been tested on general and advanced benchmarks, showing superior performance in terms of search efficiency and problem-solving capability.
  arXiv  Detail & Related papers  (2024-10-03T18:12:29Z)
• Self-Discover: Large Language Models Self-Compose Reasoning Structures [136.48389510481758]
  We introduce SELF-DISCOVER, a framework for self-discovering task-intrinsic reasoning structures. SELF-DISCOVER substantially improves GPT-4 and PaLM 2's performance on challenging reasoning benchmarks. We show that the self-discovered reasoning structures are universally applicable across model families.
  arXiv  Detail & Related papers  (2024-02-06T01:13:53Z)
• Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data? [49.688233418425995]
  Struc-Bench is a comprehensive benchmark featuring prominent Large Language Models (LLMs) We propose two innovative metrics, P-Score (Prompting Score) and H-Score (Heuristical Score) Our experiments show that applying our structure-aware fine-tuning to LLaMA-7B leads to substantial performance gains.
  arXiv  Detail & Related papers  (2023-09-16T11:31:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.