Towards One-Shot Learning for Text Classification using Inductive Logic
Programming
- URL: http://arxiv.org/abs/2308.15885v1
- Date: Wed, 30 Aug 2023 09:04:06 GMT
- Title: Towards One-Shot Learning for Text Classification using Inductive Logic
Programming
- Authors: Ghazal Afroozi Milani (University of Surrey), Daniel Cyrus (University
of Surrey), Alireza Tamaddoni-Nezhad (University of Surrey)
- Abstract summary: In this paper, we explore an Inductive Logic Programming approach for one-shot text classification.
Results indicate that MIL can learn text classification rules from a small number of training examples.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: With the ever-increasing potential of AI to perform personalised tasks, it is
becoming essential to develop new machine learning techniques which are
data-efficient and do not require hundreds or thousands of training data. In
this paper, we explore an Inductive Logic Programming approach for one-shot
text classification. In particular, we explore the framework of
Meta-Interpretive Learning (MIL), along with using common-sense background
knowledge extracted from ConceptNet. Results indicate that MIL can learn text
classification rules from a small number of training examples. Moreover, the
higher complexity of chosen examples, the higher accuracy of the outcome.
Related papers
- Smart Expert System: Large Language Models as Text Classifiers [3.218954041700146]
This paper introduces the Smart Expert System, a novel approach that leverages Large Language Models (LLMs) as text classifiers.
The system simplifies the traditional text classification workflow, eliminating the need for extensive preprocessing and domain expertise.
It is shown that the system's performance can be further enhanced through few-shot or fine-tuning strategies.
arXiv Detail & Related papers (2024-05-17T04:05:05Z) - Harnessing the Power of Beta Scoring in Deep Active Learning for
Multi-Label Text Classification [6.662167018900634]
Our study introduces a novel deep active learning strategy, capitalizing on the Beta family of proper scoring rules within the Expected Loss Reduction framework.
It computes the expected increase in scores using the Beta Scoring Rules, which are then transformed into sample vector representations.
Comprehensive evaluations across both synthetic and real datasets reveal our method's capability to often outperform established acquisition techniques in multi-label text classification.
arXiv Detail & Related papers (2024-01-15T00:06:24Z) - Language models are weak learners [71.33837923104808]
We show that prompt-based large language models can operate effectively as weak learners.
We incorporate these models into a boosting approach, which can leverage the knowledge within the model to outperform traditional tree-based boosting.
Results illustrate the potential for prompt-based LLMs to function not just as few-shot learners themselves, but as components of larger machine learning pipelines.
arXiv Detail & Related papers (2023-06-25T02:39:19Z) - Prompt-Learning for Fine-Grained Entity Typing [40.983849729537795]
We investigate the application of prompt-learning on fine-grained entity typing in fully supervised, few-shot and zero-shot scenarios.
We propose a self-supervised strategy that carries out distribution-level optimization in prompt-learning to automatically summarize the information of entity types.
arXiv Detail & Related papers (2021-08-24T09:39:35Z) - SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation.
Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
arXiv Detail & Related papers (2021-01-02T01:15:57Z) - Information Theoretic Meta Learning with Gaussian Processes [74.54485310507336]
We formulate meta learning using information theoretic concepts; namely, mutual information and the information bottleneck.
By making use of variational approximations to the mutual information, we derive a general and tractable framework for meta learning.
arXiv Detail & Related papers (2020-09-07T16:47:30Z) - DeCLUTR: Deep Contrastive Learning for Unsupervised Textual
Representations [4.36561468436181]
We present DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations.
Our approach closes the performance gap between unsupervised and supervised pretraining for universal sentence encoders.
Our code and pretrained models are publicly available and can be easily adapted to new domains or used to embed unseen text.
arXiv Detail & Related papers (2020-06-05T20:00:28Z) - Pre-training Text Representations as Meta Learning [113.3361289756749]
We introduce a learning algorithm which directly optimize model's ability to learn text representations for effective learning of downstream tasks.
We show that there is an intrinsic connection between multi-task pre-training and model-agnostic meta-learning with a sequence of meta-train steps.
arXiv Detail & Related papers (2020-04-12T09:05:47Z) - Knowledge Guided Metric Learning for Few-Shot Text Classification [22.832467388279873]
We propose to introduce external knowledge into few-shot learning to imitate human knowledge.
Inspired by human intelligence, we propose to introduce external knowledge into few-shot learning to imitate human knowledge.
We demonstrate that our method outperforms the state-of-the-art few-shot text classification models.
arXiv Detail & Related papers (2020-04-04T10:56:26Z) - Provable Meta-Learning of Linear Representations [114.656572506859]
We provide fast, sample-efficient algorithms to address the dual challenges of learning a common set of features from multiple, related tasks, and transferring this knowledge to new, unseen tasks.
We also provide information-theoretic lower bounds on the sample complexity of learning these linear features.
arXiv Detail & Related papers (2020-02-26T18:21:34Z) - Exploring the Limits of Transfer Learning with a Unified Text-to-Text
Transformer [64.22926988297685]
Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP)
In this paper, we explore the landscape of introducing transfer learning techniques for NLP by a unified framework that converts all text-based language problems into a text-to-text format.
arXiv Detail & Related papers (2019-10-23T17:37:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.