Bridging Expert Knowledge and Automated Feature Engineering via Self-Evolution

• URL: http://arxiv.org/abs/2606.08800v1
• Date: Sun, 07 Jun 2026 19:39:03 GMT
• Title: Bridging Expert Knowledge and Automated Feature Engineering via Self-Evolution
• Abstract summary: We present FEST (Feature Engineering with Self-evolving Trees), combining dual-stream feature generation (semantic and deterministic), semantic deduplication, and tree-guided iterative evolution to discover auditable features from raw text and images.<n>An evaluation shows FEST achieves 60-80% coverage of expert-designed brand features at strict semantic-alignment thresholds, corroborated by a human expert study rating features highly on relevance, clarity, and actionability.<n>To enable systematic evaluation of expert alignment in automated feature engineering, we release BrandGuide, the first dataset pairing expert-designed features with 1M+ assets across 2,683 brands
• Score: 35.18494473127365
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In high-stakes settings such as brand compliance, clinical care, and content moderation, machine learning cannot be deployed as opaque oracles: practitioners inspect the features driving model decisions, and models must leverage the expert documentation governing these domains. In practice, the data arrives as unstructured content, and features extracted from it must be interpretable, discriminative, and aligned with what experts consider important. Existing methods fall short: they target tabular inputs, lack demonstrated expert alignment, and cannot operationalize qualitative criteria such as 'maintain professional tone' into precise features. We present FEST (Feature Engineering with Self-evolving Trees), combining dual-stream feature generation (semantic and deterministic), semantic deduplication, and tree-guided iterative evolution to discover auditable features from raw text and images. FEST leads in 17 of 20 classifier-task combinations across brand classification, content authenticity detection, and stress detection, with a mean gain of 4.2 pp over the strongest baseline across five classifiers. An LLM-as-judge evaluation shows FEST achieves 60-80% coverage of expert-designed brand features at strict semantic-alignment thresholds, corroborated by a human expert study rating features highly on relevance, clarity, and actionability. When seeded with expert guidelines, FEST refines qualitative criteria into operational features, improving accuracy by 6-12 pp on average across brands. To enable systematic evaluation of expert alignment in automated feature engineering, we release BrandGuide, the first dataset pairing expert-designed features with 1M+ assets across 2,683 brands. By grounding feature engineering in expert knowledge, FEST opens a practical pathway for interpretable ML in domains demanding human oversight.

Related papers

• From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agents [69.07911499189129]
  We introduce KAPRO, a framework that evaluates cognitive-behavioral alignment by decoupling an agent's metacognitive judgment (Knowing) from its spontaneous execution (Acting)<n>Experiments show that self-awareness capability is strongly correlated with task success but degrades sharply in internal-capability settings.<n> proprietary and reasoning-oriented models demonstrate more reliable cognitive gating.
  arXiv  Detail & Related papers  (2026-06-09T17:17:08Z)
• Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation [32.5154721488471]
  Large Language Models (LLMs) exhibit plateauing performance on conventional benchmarks.<n>Existing frameworks suffer from narrow domain coverage, reliance on generalist tasks, or self-evaluation biases.<n>We present XpertBench, a high-fidelity benchmark engineered to assess LLMs across authentic professional domains.
  arXiv  Detail & Related papers  (2026-03-27T11:28:15Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• DepthCharge: A Domain-Agnostic Framework for Measuring Depth-Dependent Knowledge in Large Language Models [51.56484100374058]
  Large Language Models appear competent when answering general questions but often fail when pushed into domain-specific details.<n>We present DepthCharge, a domain-agnostic framework that measures knowledge depth through three innovations.<n> adaptive probing that generates follow-up questions based on concepts the model actually mentions, on-demand fact verification from authoritative sources, and survival statistics with constant sample sizes at every depth level.
  arXiv  Detail & Related papers  (2026-03-05T20:49:11Z)
• Optimizing In-Context Demonstrations for LLM-based Automated Grading [31.353360036776976]
  GUIDE (Grading Using Iteratively Designed Exemplars) is a framework that reframes exemplar selection and refinement as a boundary-focused optimization problem.<n>We show that GUIDE significantly outperforms standard retrieval baselines in experiments in physics, chemistry, and pedagogical content knowledge.
  arXiv  Detail & Related papers  (2026-02-28T04:52:38Z)
• Encyclo-K: Evaluating LLMs with Dynamically Composed Knowledge Statements [78.87065404966002]
  Existing benchmarks predominantly curate questions at the question level.<n>We propose Encyclo-K, a statement-based benchmark that rethinks benchmark construction from the ground up.
  arXiv  Detail & Related papers  (2025-12-31T13:55:54Z)
• Data Annotation Quality Problems in AI-Enabled Perception System Development [3.716862357836751]
  Data annotation is essential but highly error-prone in the development of AI-enabled perception systems.<n>We develop a taxonomy of 18 recurring annotation error types across three data-quality dimensions.<n>This study contributes to SE4AI by offering a shared vocabulary, diagnostic toolset, and actionable guidance for building trustworthy AI-enabled perception systems.
  arXiv  Detail & Related papers  (2025-11-20T14:30:51Z)
• OutboundEval: A Dual-Dimensional Benchmark for Expert-Level Intelligent Outbound Evaluation of Xbench's Professional-Aligned Series [36.88936933010042]
  OutboundEval is a comprehensive benchmark for evaluating large language models (LLMs) in intelligent outbound calling scenarios.<n>We design a benchmark spanning six major business domains and 30 representative sub-scenarios, each with scenario-specific process decomposition, weighted scoring, and domain-adaptive metrics.<n>We introduce a dynamic evaluation method that adapts to task variations, integrating automated and human-in-the-loop assessment to measure task execution accuracy, professional knowledge application, adaptability, and user experience quality.
  arXiv  Detail & Related papers  (2025-10-24T08:27:58Z)
• Uncertainty-Driven Expert Control: Enhancing the Reliability of Medical Vision-Language Models [52.2001050216955]
  Existing methods aim to enhance the performance of Medical Vision Language Model (MedVLM) by adjusting model structure, fine-tuning with high-quality data, or through preference fine-tuning.<n>We propose an expert-in-the-loop framework named Expert-Controlled-Free Guidance (Expert-CFG) to align MedVLM with clinical expertise without additional training.
  arXiv  Detail & Related papers  (2025-07-12T09:03:30Z)
• ExpertGenQA: Open-ended QA generation in Specialized Domains [9.412082058055823]
  ExpertGenQA is a protocol that combines few-shot learning with structured topic and style categorization to generate comprehensive domain-specific QA pairs.<n>We show that ExpertGenQA achieves twice the efficiency of baseline few-shot approaches while maintaining $94.4%$ topic coverage.
  arXiv  Detail & Related papers  (2025-03-04T19:09:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.