Retrieval-Augmented Feature Generation for Domain-Specific Classification

• URL: http://arxiv.org/abs/2406.11177v2
• Date: Sat, 28 Dec 2024 21:16:38 GMT
• Title: Retrieval-Augmented Feature Generation for Domain-Specific Classification
• Authors: Xinhao Zhang, Jinghan Zhang, Fengran Mo, Yuzhong Chen, Kunpeng Liu,
• Abstract summary: This paper introduces a new method RAFG for generating reasonable and explainable features specific to domain classification tasks.<n>To generate new features with interpretability in domain knowledge, we perform information retrieval on existing features to identify potential feature associations.<n>We develop a Large Language Model (LLM)-based framework for feature generation with reasoning to verify and filter features during the generation process.
• Score: 7.445440204397416
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Feature generation can significantly enhance learning outcomes, particularly for tasks with limited data. An effective way to improve feature generation is by expanding the current feature space using existing features and enriching the informational content. However, generating new, interpretable features in application fields often requires domain-specific knowledge about the existing features. This paper introduces a new method RAFG for generating reasonable and explainable features specific to domain classification tasks. To generate new features with interpretability in domain knowledge, we perform information retrieval on existing features to identify potential feature associations, and utilize these associations to generate meaningful features. Furthermore, we develop a Large Language Model (LLM)-based framework for feature generation with reasoning to verify and filter features during the generation process. Experiments across several datasets in medical, economic, and geographic domains show that our RAFG method produces high-quality, meaningful features and significantly improves classification performance compared with baseline methods.

Related papers

• Task-Agnostic Guided Feature Expansion for Class-Incremental Learning [59.78858949137853]
  The ability to learn new concepts while preserve the learned knowledge is desirable for learning systems in Class-Incremental Learning (CIL) Recently, feature expansion of the model become a prevalent solution for CIL, where the old features are fixed during the training of the new task while new features are expanded for the new tasks. We propose a framework called Task-Agnostic Guided Feature Expansion (TagFex) to promote the learning and transferring of diverse features across tasks.
  arXiv  Detail & Related papers  (2025-03-02T09:56:50Z)
• Developing Retrieval Augmented Generation (RAG) based LLM Systems from PDFs: An Experience Report [3.4632900249241874]
  This paper presents an experience report on the development of Retrieval Augmented Generation (RAG) systems using PDF documents as the primary data source. The RAG architecture combines generative capabilities of Large Language Models (LLMs) with the precision of information retrieval. The practical implications of this research lie in enhancing the reliability of generative AI systems in various sectors.
  arXiv  Detail & Related papers  (2024-10-21T12:21:49Z)
• Augmented Functional Random Forests: Classifier Construction and Unbiased Functional Principal Components Importance through Ad-Hoc Conditional Permutations [0.0]
  This paper introduces a novel supervised classification strategy that integrates functional data analysis with tree-based methods. We propose augmented versions of functional classification trees and functional random forests, incorporating a new tool for assessing the importance of functional principal components.
  arXiv  Detail & Related papers  (2024-08-23T15:58:41Z)
• Prospector Heads: Generalized Feature Attribution for Large Models & Data [82.02696069543454]
  We introduce prospector heads, an efficient and interpretable alternative to explanation-based attribution methods. We demonstrate how prospector heads enable improved interpretation and discovery of class-specific patterns in input data.
  arXiv  Detail & Related papers  (2024-02-18T23:01:28Z)
• Text2Data: Low-Resource Data Generation with Textual Control [104.38011760992637]
  Natural language serves as a common and straightforward control signal for humans to interact seamlessly with machines. We propose Text2Data, a novel approach that utilizes unlabeled data to understand the underlying data distribution through an unsupervised diffusion model. It undergoes controllable finetuning via a novel constraint optimization-based learning objective that ensures controllability and effectively counteracts catastrophic forgetting.
  arXiv  Detail & Related papers  (2024-02-08T03:41:39Z)
• GPT Struct Me: Probing GPT Models on Narrative Entity Extraction [2.049592435988883]
  We evaluate the capabilities of two state-of-the-art language models -- GPT-3 and GPT-3.5 -- in the extraction of narrative entities. This study is conducted on the Text2Story Lusa dataset, a collection of 119 Portuguese news articles.
  arXiv  Detail & Related papers  (2023-11-24T16:19:04Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• Empower Text-Attributed Graphs Learning with Large Language Models (LLMs) [5.920353954082262]
  We propose a plug-and-play approach to empower text-attributed graphs through node generation using Large Language Models (LLMs) We employ an edge predictor to capture the structural information inherent in the raw dataset and integrate the newly generated samples into the original graph. Experiments demonstrate the outstanding performance of our proposed paradigm, particularly in low-shot scenarios.
  arXiv  Detail & Related papers  (2023-10-15T16:04:28Z)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [51.90524745663737]
  A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks. Our method achieves state-of-the-art results on well-established TAG datasets. Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
  arXiv  Detail & Related papers  (2023-05-31T03:18:03Z)
• Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy [164.83371924650294]
  We show that strong performance can be achieved by a method we call Iter-RetGen, which synergizes retrieval and generation in an iterative manner. A model output shows what might be needed to finish a task, and thus provides an informative context for retrieving more relevant knowledge. Iter-RetGen processes all retrieved knowledge as a whole and largely preserves the flexibility in generation without structural constraints.
  arXiv  Detail & Related papers  (2023-05-24T16:17:36Z)
• Challenges of ELA-guided Function Evolution using Genetic Programming [0.0]
  We show that a genetic programming approach guided by exploratory landscape analysis (ELA) properties is not always able to find satisfying functions. Our results suggest that careful considerations of the weighting of landscape properties, as well as the distance measure used, might be required to evolve functions that are sufficiently representative to the target landscape.
  arXiv  Detail & Related papers  (2023-05-24T15:31:01Z)
• Exploring In-Context Learning Capabilities of Foundation Models for Generating Knowledge Graphs from Text [3.114960935006655]
  This paper aims to improve the state of the art of automatic construction and completion of knowledge graphs from text. In this context, one emerging paradigm is in-context learning where a language model is used as it is with a prompt.
  arXiv  Detail & Related papers  (2023-05-15T17:10:19Z)
• Active Retrieval Augmented Generation [123.68874416084499]
  Augmenting large language models (LMs) by retrieving information from external knowledge resources is one promising solution. Most existing retrieval augmented LMs employ a retrieve-and-generate setup that only retrieves information once based on the input. We propose Forward-Looking Active REtrieval augmented generation (FLARE), a generic method which iteratively uses a prediction of the upcoming sentence to anticipate future content.
  arXiv  Detail & Related papers  (2023-05-11T17:13:40Z)
• Feature construction using explanations of individual predictions [0.0]
  We propose a novel approach for reducing the search space based on aggregation of instance-based explanations of predictive models. We empirically show that reducing the search to these groups significantly reduces the time of feature construction. We show significant improvements in classification accuracy for several classifiers and demonstrate the feasibility of the proposed feature construction even for large datasets.
  arXiv  Detail & Related papers  (2023-01-23T18:59:01Z)
• Informative Text Generation from Knowledge Triples [56.939571343797304]
  We propose a novel memory augmented generator that employs a memory network to memorize the useful knowledge learned during the training. We derive a dataset from WebNLG for our new setting and conduct extensive experiments to investigate the effectiveness of our model.
  arXiv  Detail & Related papers  (2022-09-26T14:35:57Z)
• Group-wise Reinforcement Feature Generation for Optimal and Explainable Representation Space Reconstruction [25.604176830832586]
  We reformulate representation space reconstruction into an interactive process of nested feature generation and selection. We design a group-wise generation strategy to cross a feature group, an operation, and another feature group to generate new features. We present extensive experiments to demonstrate the effectiveness, efficiency, traceability, and explicitness of our system.
  arXiv  Detail & Related papers  (2022-05-28T21:34:14Z)
• Attribute Prototype Network for Any-Shot Learning [113.50220968583353]
  We argue that an image representation with integrated attribute localization ability would be beneficial for any-shot, i.e. zero-shot and few-shot, image classification tasks. We propose a novel representation learning framework that jointly learns global and local features using only class-level attributes.
  arXiv  Detail & Related papers  (2022-04-04T02:25:40Z)
• Structure-Aware Feature Generation for Zero-Shot Learning [108.76968151682621]
  We introduce a novel structure-aware feature generation scheme, termed as SA-GAN, to account for the topological structure in learning both the latent space and the generative networks. Our method significantly enhances the generalization capability on unseen-classes and consequently improve the classification performance.
  arXiv  Detail & Related papers  (2021-08-16T11:52:08Z)
• ENT-DESC: Entity Description Generation by Exploring Knowledge Graph [53.03778194567752]
  In practice, the input knowledge could be more than enough, since the output description may only cover the most significant knowledge. We introduce a large-scale and challenging dataset to facilitate the study of such a practical scenario in KG-to-text. We propose a multi-graph structure that is able to represent the original graph information more comprehensively.
  arXiv  Detail & Related papers  (2020-04-30T14:16:19Z)
• Global Context-Aware Progressive Aggregation Network for Salient Object Detection [117.943116761278]
  We propose a novel network named GCPANet to integrate low-level appearance features, high-level semantic features, and global context features. We show that the proposed approach outperforms the state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-03-02T04:26:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.