How would Stance Detection Techniques Evolve after the Launch of ChatGPT?

• URL: http://arxiv.org/abs/2212.14548v4
• Date: Mon, 12 Aug 2024 03:44:48 GMT
• Title: How would Stance Detection Techniques Evolve after the Launch of ChatGPT?
• Authors: Bowen Zhang, Daijun Ding, Liwen Jing, Genan Dai, Nan Yin,
• Abstract summary: A new pre-trained language model chatGPT was launched on Nov 30, 2022. ChatGPT can achieve SOTA or similar performance for commonly used datasets including SemEval-2016 and P-Stance. ChatGPT has the potential to be the best AI model for stance detection tasks in NLP.
• Score: 5.756359016880821
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Stance detection refers to the task of extracting the standpoint (Favor, Against or Neither) towards a target in given texts. Such research gains increasing attention with the proliferation of social media contents. The conventional framework of handling stance detection is converting it into text classification tasks. Deep learning models have already replaced rule-based models and traditional machine learning models in solving such problems. Current deep neural networks are facing two main challenges which are insufficient labeled data and information in social media posts and the unexplainable nature of deep learning models. A new pre-trained language model chatGPT was launched on Nov 30, 2022. For the stance detection tasks, our experiments show that ChatGPT can achieve SOTA or similar performance for commonly used datasets including SemEval-2016 and P-Stance. At the same time, ChatGPT can provide explanation for its own prediction, which is beyond the capability of any existing model. The explanations for the cases it cannot provide classification results are especially useful. ChatGPT has the potential to be the best AI model for stance detection tasks in NLP, or at least change the research paradigm of this field. ChatGPT also opens up the possibility of building explanatory AI for stance detection.

Related papers

• Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines [83.65380507372483]
  Large pre-trained models can dramatically reduce the amount of task-specific data required to solve a problem, but they often fail to capture domain-specific nuances out of the box. This paper shows how to leverage recent advances in NLP and multi-modal learning to augment a pre-trained model with search engine retrieval.
  arXiv  Detail & Related papers  (2023-11-29T05:33:28Z)
• Chatbots Are Not Reliable Text Annotators [0.0]
  ChatGPT is a closed-source product which has major drawbacks with regards to transparency, cost, and data protection. Recent advances in open-source (OS) large language models (LLMs) offer alternatives which remedy these challenges.
  arXiv  Detail & Related papers  (2023-11-09T22:28:14Z)
• Can You Follow Me? Testing Situational Understanding in ChatGPT [17.52769657390388]
  "situational understanding" (SU) is a critical ability for human-like AI agents. We propose a novel synthetic environment for SU testing in chat-oriented models. We find that despite the fundamental simplicity of the task, the model's performance reflects an inability to retain correct environment states.
  arXiv  Detail & Related papers  (2023-10-24T19:22:01Z)
• Can ChatGPT's Responses Boost Traditional Natural Language Processing? [12.456183060562317]
  ChatGPT has shown the potential of emerging capabilities to solve problems, without being particularly trained to solve. Previous work demonstrated these emerging capabilities in affective computing tasks. We extend this by exploring if ChatGPT has novel knowledge that would enhance existing specialised models when they are fused together.
  arXiv  Detail & Related papers  (2023-07-06T15:42:05Z)
• Investigating Chain-of-thought with ChatGPT for Stance Detection on Social Media [6.980802590994858]
  Chain-of-Thought (CoT) approach not requiring backpropagation training has emerged as a promising alternative. This paper examines CoT's effectiveness in stance detection tasks, demonstrating its superior accuracy and discussing associated challenges.
  arXiv  Detail & Related papers  (2023-04-06T14:12:02Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data [101.63682141248069]
  Chat models, such as ChatGPT, have shown impressive capabilities and have been rapidly adopted across numerous domains. We propose a pipeline that can automatically generate a high-quality multi-turn chat corpus by leveraging ChatGPT. We employ parameter-efficient tuning to enhance LLaMA, an open-source large language model.
  arXiv  Detail & Related papers  (2023-04-03T17:59:09Z)
• Will Affective Computing Emerge from Foundation Models and General AI? A First Evaluation on ChatGPT [12.456183060562317]
  ChatGPT has demonstrated competent performance across many natural language processing tasks. We evaluate the capabilities of ChatGPT to perform text classification on three affective computing problems.
  arXiv  Detail & Related papers  (2023-03-03T16:11:37Z)
• Is ChatGPT a General-Purpose Natural Language Processing Task Solver? [113.22611481694825]
  Large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot. Recently, the debut of ChatGPT has drawn a great deal of attention from the natural language processing (NLP) community. It is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot.
  arXiv  Detail & Related papers  (2023-02-08T09:44:51Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)
• Robust Conversational AI with Grounded Text Generation [77.56950706340767]
  GTG is a hybrid model which uses a large-scale Transformer neural network as its backbone. It generates responses grounded in dialog belief state and real-world knowledge for task completion.
  arXiv  Detail & Related papers  (2020-09-07T23:49:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.