LLM-Driven Reasoning for Constraint-Aware Feature Selection in Industrial Systems

• URL: http://arxiv.org/abs/2603.24979v1
• Date: Thu, 26 Mar 2026 03:10:33 GMT
• Title: LLM-Driven Reasoning for Constraint-Aware Feature Selection in Industrial Systems
• Abstract summary: We propose Model Feature Agent (MoFA), a model-driven framework that performs sequential, reasoning-based feature selection.<n>MoFA incorporates feature definitions, importance scores, correlations, and metadata into structured prompts and selects features through interpretable, constraint-aware reasoning.<n>We evaluate MoFA in three real-world industrial applications: True Interest and Time-Worthiness Prediction, Value Model Enhancement, and Notification Behavior Prediction.
• Score: 31.965000184715233
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Feature selection is a crucial step in large-scale industrial machine learning systems, directly affecting model accuracy, efficiency, and maintainability. Traditional feature selection methods rely on labeled data and statistical heuristics, making them difficult to apply in production environments where labeled data are limited and multiple operational constraints must be satisfied. To address this, we propose Model Feature Agent (MoFA), a model-driven framework that performs sequential, reasoning-based feature selection using both semantic and quantitative feature information. MoFA incorporates feature definitions, importance scores, correlations, and metadata (e.g., feature groups or types) into structured prompts and selects features through interpretable, constraint-aware reasoning. We evaluate MoFA in three real-world industrial applications: (1) True Interest and Time-Worthiness Prediction, where it improves accuracy while reducing feature group complexity, (2) Value Model Enhancement, where it discovers high-order interaction terms that yield substantial engagement gains in online experiments, and (3) Notification Behavior Prediction, where it selects compact, high-value feature subsets that improve both model accuracy and inference efficiency. Together, these results demonstrate the practicality and effectiveness of LLM-based reasoning for feature selection in real production systems.

Related papers

• A unified framework for global and local interpretability using adaptive derivative-ordered random explanation [2.6717072353469997]
  Post-hoc interpretability of complex machine learning models is of paramount importance, especially in real-world high-stakes domains such as healthcare and finance.<n>This paper provides a novel method - Adaptive Derivative-Ordered Random Explanation (ADORE) - that leverages first- and second-order derivatives to accommodate nonlinear model complexities.<n>ADORE integrates global feature importance with local sample contributions, precisely quantifying feature impact by capturing both magnitude and direction.<n>It achieves computational efficiency through randomized singular value decomposition (SVD) and dynamic sparsity detection, making it scalable to large, high-dimensional datasets.
  arXiv  Detail & Related papers  (2026-09-15T13:34:10Z)
• From Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models [73.72877445629383]
  Interpretability-Guided Data Selection (IGDS) is a framework that first identifies causal task features through frequency recall and interventional filtering.<n>We validate IGDS on mathematical reasoning, summarization, and translation tasks within Gemma-2, LLaMA-3.1, and Qwen3 models.
  arXiv  Detail & Related papers  (2026-04-28T03:16:24Z)
• LLM-MLFFN: Multi-Level Autonomous Driving Behavior Feature Fusion via Large Language Model [8.63792214154021]
  This paper presents LLM-MLFFN, a novel large language model (LLM)-enhanced multi-level feature fusion network.<n>The proposed LLM-MLFFN framework integrates priors from largescale pre-trained models and employs a multi-level approach to enhance classification accuracy.<n> Evaluation on the open trajectory dataset demonstrates the superior performance of the proposed LLM-MLFFN, achieving a classification accuracy of over 94%.
  arXiv  Detail & Related papers  (2026-03-03T02:26:04Z)
• ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuning [51.133569963553576]
  ssToken is a Self-modulated and Semantic-aware Token Selection approach.<n>We show that both self-modulated selection and semantic-aware selection alone outperform full-data fine-tuning.
  arXiv  Detail & Related papers  (2025-10-21T03:21:04Z)
• PB-IAD: Utilizing multimodal foundation models for semantic industrial anomaly detection in dynamic manufacturing environments [0.0]
  This paper presents PB-IAD (Prompt-based Industrial Anomaly Detection), a novel framework for anomaly detection.<n>It addresses three key requirements of dynamic production environments: data sparsity, agile adaptability, and domain user centricity.<n>It is benchmarked to state-of-the-art methods for anomaly detection such as PatchCore.
  arXiv  Detail & Related papers  (2025-08-20T07:53:13Z)
• FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Models [79.41859481668618]
  Large Language Models (LLMs) have significantly advanced the fact-checking studies.<n>Existing automated fact-checking evaluation methods rely on static datasets and classification metrics.<n>We introduce FACT-AUDIT, an agent-driven framework that adaptively and dynamically assesses LLMs' fact-checking capabilities.
  arXiv  Detail & Related papers  (2025-02-25T07:44:22Z)
• SELF: Surrogate-light Feature Selection with Large Language Models in Deep Recommender Systems [51.09233156090496]
  SurrogatE-Light Feature selection method for deep recommender systems.<n> SELF integrates semantic reasoning from Large Language Models with task-specific learning from surrogate models.<n> Comprehensive experiments on three public datasets from real-world recommender platforms validate the effectiveness of SELF.
  arXiv  Detail & Related papers  (2024-12-11T16:28:18Z)
• Can foundation models actively gather information in interactive environments to test hypotheses? [43.42688356541211]
  Foundation models excel at single-turn reasoning but struggle with multi-turn exploration in dynamic environments.<n>We evaluated these models on their ability to learn from experience, adapt, and gather information.
  arXiv  Detail & Related papers  (2024-12-09T12:27:21Z)
• Latent Feature Mining for Predictive Model Enhancement with Large Language Models [2.6334346517416876]
  We introduce an effective approach to formulate latent feature mining as text-to-text propositional logical reasoning. We propose FLAME, a framework that leverages large language models (LLMs) to augment observed features with latent features. We validate our framework with two case studies: the criminal justice system and the healthcare domain.
  arXiv  Detail & Related papers  (2024-10-06T03:51:32Z)
• LLM-Select: Feature Selection with Large Language Models [64.5099482021597]
  Large language models (LLMs) are capable of selecting the most predictive features, with performance rivaling the standard tools of data science.<n>Our findings suggest that LLMs may be useful not only for selecting the best features for training but also for deciding which features to collect in the first place.
  arXiv  Detail & Related papers  (2024-07-02T22:23:40Z)
• REFRESH: Responsible and Efficient Feature Reselection Guided by SHAP Values [17.489279048199304]
  REFRESH is a method to reselect features so that additional constraints that are desirable towards model performance can be achieved without having to train several new models. REFRESH's underlying algorithm is a novel technique using SHAP values and correlation analysis that can approximate for the predictions of a model without having to train these models.
  arXiv  Detail & Related papers  (2024-03-13T18:06:43Z)
• Variable Importance Matching for Causal Inference [73.25504313552516]
  We describe a general framework called Model-to-Match that achieves these goals. Model-to-Match uses variable importance measurements to construct a distance metric. We operationalize the Model-to-Match framework with LASSO.
  arXiv  Detail & Related papers  (2023-02-23T00:43:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.