Few-Shot Stance Detection via Target-Aware Prompt Distillation
- URL: http://arxiv.org/abs/2206.13214v1
- Date: Mon, 27 Jun 2022 12:04:14 GMT
- Title: Few-Shot Stance Detection via Target-Aware Prompt Distillation
- Authors: Yan Jiang, Jinhua Gao, Huawei Shen, Xueqi Cheng
- Abstract summary: This paper is inspired by the potential capability of pre-trained language models (PLMs) serving as knowledge bases and few-shot learners.
PLMs can provide essential contextual information for the targets and enable few-shot learning via prompts.
Considering the crucial role of the target in stance detection task, we design target-aware prompts and propose a novel verbalizer.
- Score: 48.40269795901453
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Stance detection aims to identify whether the author of a text is in favor
of, against, or neutral to a given target. The main challenge of this task
comes two-fold: few-shot learning resulting from the varying targets and the
lack of contextual information of the targets. Existing works mainly focus on
solving the second issue by designing attention-based models or introducing
noisy external knowledge, while the first issue remains under-explored. In this
paper, inspired by the potential capability of pre-trained language models
(PLMs) serving as knowledge bases and few-shot learners, we propose to
introduce prompt-based fine-tuning for stance detection. PLMs can provide
essential contextual information for the targets and enable few-shot learning
via prompts. Considering the crucial role of the target in stance detection
task, we design target-aware prompts and propose a novel verbalizer. Instead of
mapping each label to a concrete word, our verbalizer maps each label to a
vector and picks the label that best captures the correlation between the
stance and the target. Moreover, to alleviate the possible defect of dealing
with varying targets with a single hand-crafted prompt, we propose to distill
the information learned from multiple prompts. Experimental results show the
superior performance of our proposed model in both full-data and few-shot
scenarios.
Related papers
- Mutually-Aware Feature Learning for Few-Shot Object Counting [20.623402944601775]
Few-shot object counting has garnered significant attention for its practicality as it aims to count target objects in a query image based on given exemplars without the need for additional training.
We propose a novel framework, Mutually-Aware FEAture learning(MAFEA), which encodes query and exemplar features mutually aware of each other from the outset.
Our model reaches a new state-of-the-art performance on the two challenging benchmarks, FSCD-LVIS and FSC-147, with a remarkably reduced degree of the target confusion problem.
arXiv Detail & Related papers (2024-08-19T06:46:24Z) - Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detection [101.15777242546649]
Open vocabulary object detection (OVD) aims at seeking an optimal object detector capable of recognizing objects from both base and novel categories.
Recent advances leverage knowledge distillation to transfer insightful knowledge from pre-trained large-scale vision-language models to the task of object detection.
We present a novel OVD framework termed LBP to propose learning background prompts to harness explored implicit background knowledge.
arXiv Detail & Related papers (2024-06-01T17:32:26Z) - Single-Shot and Multi-Shot Feature Learning for Multi-Object Tracking [55.13878429987136]
We propose a simple yet effective two-stage feature learning paradigm to jointly learn single-shot and multi-shot features for different targets.
Our method has achieved significant improvements on MOT17 and MOT20 datasets while reaching state-of-the-art performance on DanceTrack dataset.
arXiv Detail & Related papers (2023-11-17T08:17:49Z) - Stance Detection with Collaborative Role-Infused LLM-Based Agents [39.75103353173015]
Stance detection is vital for content analysis in web and social media research.
However, stance detection requires advanced reasoning to infer authors' implicit viewpoints.
We design a three-stage framework in which LLMs are designated distinct roles.
We achieve state-of-the-art performance across multiple datasets.
arXiv Detail & Related papers (2023-10-16T14:46:52Z) - Memory Injections: Correcting Multi-Hop Reasoning Failures during
Inference in Transformer-Based Language Models [4.343604069244352]
We propose an approach to pinpoint and rectify multi-hop reasoning failures through targeted memory injections on attention heads.
We show that a simple, efficient, and targeted memory injection into a key attention layer can often increase the probability of the desired next token in multi-hop tasks, by up to 424%.
arXiv Detail & Related papers (2023-09-11T16:39:30Z) - Contextual information integration for stance detection via
cross-attention [59.662413798388485]
Stance detection deals with identifying an author's stance towards a target.
Most existing stance detection models are limited because they do not consider relevant contextual information.
We propose an approach to integrate contextual information as text.
arXiv Detail & Related papers (2022-11-03T15:04:29Z) - Zero-shot stance detection based on cross-domain feature enhancement by
contrastive learning [9.719309795292773]
We propose a stance detection approach that can efficiently adapt to unseen targets.
We first augment the data by masking the topic words of sentences.
We then feed the augmented data to an unsupervised contrastive learning module to capture transferable features.
arXiv Detail & Related papers (2022-10-07T07:45:40Z) - Language-guided Navigation via Cross-Modal Grounding and Alternate
Adversarial Learning [66.9937776799536]
The emerging vision-and-language navigation (VLN) problem aims at learning to navigate an agent to the target location in unseen photo-realistic environments.
The main challenges of VLN arise mainly from two aspects: first, the agent needs to attend to the meaningful paragraphs of the language instruction corresponding to the dynamically-varying visual environments.
We propose a cross-modal grounding module to equip the agent with a better ability to track the correspondence between the textual and visual modalities.
arXiv Detail & Related papers (2020-11-22T09:13:46Z) - Probing Task-Oriented Dialogue Representation from Language Models [106.02947285212132]
This paper investigates pre-trained language models to find out which model intrinsically carries the most informative representation for task-oriented dialogue tasks.
We fine-tune a feed-forward layer as the classifier probe on top of a fixed pre-trained language model with annotated labels in a supervised way.
arXiv Detail & Related papers (2020-10-26T21:34:39Z) - Leveraging Bottom-Up and Top-Down Attention for Few-Shot Object
Detection [31.1548809359908]
Few-shot object detection aims at detecting objects with few annotated examples.
We propose an attentive few-shot object detection network (AttFDNet) that takes the advantages of both top-down and bottom-up attention.
arXiv Detail & Related papers (2020-07-23T16:12:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.