Zero-Shot Stance Detection using Contextual Data Generation with LLMs

• URL: http://arxiv.org/abs/2405.11637v1
• Date: Sun, 19 May 2024 17:58:26 GMT
• Title: Zero-Shot Stance Detection using Contextual Data Generation with LLMs
• Authors: Ghazaleh Mahmoudi, Babak Behkamkia, Sauleh Eetemadi,
• Abstract summary: We propose Dynamic Model Adaptation with Contextual Data Generation (DyMoAdapt) In this approach, we aim to fine-tune an existing model at test time. We achieve this by generating new topic-specific data using GPT-3. This method could enhance performance by allowing the adaptation of the model to new topics.
• Score: 0.04096453902709291
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stance detection, the classification of attitudes expressed in a text towards a specific topic, is vital for applications like fake news detection and opinion mining. However, the scarcity of labeled data remains a challenge for this task. To address this problem, we propose Dynamic Model Adaptation with Contextual Data Generation (DyMoAdapt) that combines Few-Shot Learning and Large Language Models. In this approach, we aim to fine-tune an existing model at test time. We achieve this by generating new topic-specific data using GPT-3. This method could enhance performance by allowing the adaptation of the model to new topics. However, the results did not increase as we expected. Furthermore, we introduce the Multi Generated Topic VAST (MGT-VAST) dataset, which extends VAST using GPT-3. In this dataset, each context is associated with multiple topics, allowing the model to understand the relationship between contexts and various potential topics

Related papers

• BlendX: Complex Multi-Intent Detection with Blended Patterns [4.852816974803059]
  We present BlendX, a suite of refined datasets featuring more diverse patterns than their predecessors. For dataset construction, we utilize both rule-baseds and a generative tool -- OpenAI's ChatGPT -- which is augmented with a similarity-driven strategy for utterance selection. Experiments on BlendX reveal that state-of-the-art MID models struggle with the challenges posed by the new datasets.
  arXiv  Detail & Related papers  (2024-03-27T06:13:04Z)
• Exploiting Contextual Target Attributes for Target Sentiment Classification [53.30511968323911]
  Existing PTLM-based models for TSC can be categorized into two groups: 1) fine-tuning-based models that adopt PTLM as the context encoder; 2) prompting-based models that transfer the classification task to the text/word generation task. We present a new perspective of leveraging PTLM for TSC: simultaneously leveraging the merits of both language modeling and explicit target-context interactions via contextual target attributes.
  arXiv  Detail & Related papers  (2023-12-21T11:45:28Z)
• GPT Struct Me: Probing GPT Models on Narrative Entity Extraction [2.049592435988883]
  We evaluate the capabilities of two state-of-the-art language models -- GPT-3 and GPT-3.5 -- in the extraction of narrative entities. This study is conducted on the Text2Story Lusa dataset, a collection of 119 Portuguese news articles.
  arXiv  Detail & Related papers  (2023-11-24T16:19:04Z)
• Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias [92.41919689753051]
  Large language models (LLMs) have been recently leveraged as training data generators for various natural language processing (NLP) tasks. We investigate training data generation with diversely attributed prompts, which have the potential to yield diverse and attributed generated data. We show that attributed prompts outperform simple class-conditional prompts in terms of the resulting model's performance.
  arXiv  Detail & Related papers  (2023-06-28T03:31:31Z)
• Going beyond research datasets: Novel intent discovery in the industry setting [60.90117614762879]
  This paper proposes methods to improve the intent discovery pipeline deployed in a large e-commerce platform. We show the benefit of pre-training language models on in-domain data: both self-supervised and with weak supervision. We also devise the best method to utilize the conversational structure (i.e., question and answer) of real-life datasets during fine-tuning for clustering tasks, which we call Conv.
  arXiv  Detail & Related papers  (2023-05-09T14:21:29Z)
• Unified Visual Relationship Detection with Vision and Language Models [89.77838890788638]
  This work focuses on training a single visual relationship detector predicting over the union of label spaces from multiple datasets. We propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models. Empirical results on both human-object interaction detection and scene-graph generation demonstrate the competitive performance of our model.
  arXiv  Detail & Related papers  (2023-03-16T00:06:28Z)
• Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning [19.7066703371736]
  We propose a novel short text topic modeling framework, Topic-Semantic Contrastive Topic Model (TSCTM) Our TSCTM outperforms state-of-the-art baselines regardless of the data augmentation availability, producing high-quality topics and topic distributions.
  arXiv  Detail & Related papers  (2022-11-23T11:33:43Z)
• Data Augmentation for Intent Classification with Off-the-shelf Large Language Models [13.895236210726202]
  We propose a prompting-based approach to generate labelled training data for intent classification with off-the-shelf language models. We evaluate the proposed method in a few-shot setting on four diverse intent classification tasks.
  arXiv  Detail & Related papers  (2022-04-05T03:29:26Z)
• The Surprising Performance of Simple Baselines for Misinformation Detection [4.060731229044571]
  We examine the performance of a broad set of modern transformer-based language models. We present our framework as a baseline for creating and evaluating new methods for misinformation detection.
  arXiv  Detail & Related papers  (2021-04-14T16:25:22Z)
• Partially-Aligned Data-to-Text Generation with Distant Supervision [69.15410325679635]
  We propose a new generation task called Partially-Aligned Data-to-Text Generation (PADTG) It is more practical since it utilizes automatically annotated data for training and thus considerably expands the application domains. Our framework outperforms all baseline models as well as verify the feasibility of utilizing partially-aligned data.
  arXiv  Detail & Related papers  (2020-10-03T03:18:52Z)
• Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling [81.33107307509718]
  We propose a topic adaptive storyteller to model the ability of inter-topic generalization. We also propose a prototype encoding structure to model the ability of intra-topic derivation. Experimental results show that topic adaptation and prototype encoding structure mutually bring benefit to the few-shot model.
  arXiv  Detail & Related papers  (2020-08-11T03:55:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.