Bot or Human? Detecting ChatGPT Imposters with A Single Question

• URL: http://arxiv.org/abs/2305.06424v4
• Date: Sun, 11 Aug 2024 18:56:50 GMT
• Title: Bot or Human? Detecting ChatGPT Imposters with A Single Question
• Authors: Hong Wang, Xuan Luo, Weizhi Wang, Xifeng Yan,
• Abstract summary: Large language models (LLMs) have recently demonstrated impressive capabilities in natural language understanding and generation. There is a concern that they can be misused for malicious purposes, such as fraud or denial-of-service attacks. We propose a framework named FLAIR, Finding Large Language Model Authenticity via a Single Inquiry and Response, to detect conversational bots in an online manner.
• Score: 29.231261118782925
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) like GPT-4 have recently demonstrated impressive capabilities in natural language understanding and generation. However, there is a concern that they can be misused for malicious purposes, such as fraud or denial-of-service attacks. Therefore, it is crucial to develop methods for detecting whether the party involved in a conversation is a bot or a human. In this paper, we propose a framework named FLAIR, Finding Large Language Model Authenticity via a Single Inquiry and Response, to detect conversational bots in an online manner. Specifically, we target a single question scenario that can effectively differentiate human users from bots. The questions are divided into two categories: those that are easy for humans but difficult for bots (e.g., counting, substitution, searching, and ASCII art reasoning), and those that are easy for bots but difficult for humans (e.g., memorization and computation). Our approach shows different strengths of these questions in their effectiveness, providing a new way for online service providers to protect themselves against nefarious activities. Our code and question set are available at https://github.com/hongwang600/FLAIR.

Related papers

• Adversarial Botometer: Adversarial Analysis for Social Bot Detection [1.9280536006736573]
  Social bots produce content that mimics human creativity. Malicious social bots emerge to deceive people with their unrealistic content. We evaluate the behavior of a text-based bot detector in a competitive environment.
  arXiv  Detail & Related papers  (2024-05-03T11:28:21Z)
• Spot the bot: Coarse-Grained Partition of Semantic Paths for Bots and Humans [55.2480439325792]
  This paper focuses on comparing structures of the coarse-grained partitions of semantic paths for human-written and bot-generated texts. As the semantic structure may be different for different languages, we investigate Russian, English, German, and Vietnamese languages.
  arXiv  Detail & Related papers  (2024-02-27T10:38:37Z)
• My Brother Helps Me: Node Injection Based Adversarial Attack on Social Bot Detection [69.99192868521564]
  Social platforms such as Twitter are under siege from a multitude of fraudulent users. Due to the structure of social networks, the majority of methods are based on the graph neural network(GNN), which is susceptible to attacks. We propose a node injection-based adversarial attack method designed to deceive bot detection models.
  arXiv  Detail & Related papers  (2023-10-11T03:09:48Z)
• You are a Bot! -- Studying the Development of Bot Accusations on Twitter [1.7626250599622473]
  In the absence of ground truth data, researchers may want to tap into the wisdom of the crowd. Our research presents the first large-scale study of bot accusations on Twitter. It shows how the term bot became an instrument of dehumanization in social media conversations.
  arXiv  Detail & Related papers  (2023-02-01T16:09:11Z)
• Implementing a Chatbot Solution for Learning Management System [0.0]
  One of the main problem that chatbots face today is to mimic human language. Extreme programming methodology was chosen to use integrate ChatterBot, Pyside2, web scraping and Tampermonkey into Blackboard. We showed the plausibility of integrating an AI bot in an educational setting.
  arXiv  Detail & Related papers  (2022-06-27T11:04:42Z)
• A Deep Learning Approach to Integrate Human-Level Understanding in a Chatbot [0.4632366780742501]
  Unlike humans, chatbots can serve multiple customers at a time, are available 24/7 and reply in less than a fraction of a second. We performed sentiment analysis, emotion detection, intent classification and named-entity recognition using deep learning to develop chatbots with humanistic understanding and intelligence.
  arXiv  Detail & Related papers  (2021-12-31T22:26:41Z)
• Identification of Twitter Bots based on an Explainable ML Framework: the US 2020 Elections Case Study [72.61531092316092]
  This paper focuses on the design of a novel system for identifying Twitter bots based on labeled Twitter data. Supervised machine learning (ML) framework is adopted using an Extreme Gradient Boosting (XGBoost) algorithm. Our study also deploys Shapley Additive Explanations (SHAP) for explaining the ML model predictions.
  arXiv  Detail & Related papers  (2021-12-08T14:12:24Z)
• CheerBots: Chatbots toward Empathy and Emotionusing Reinforcement Learning [60.348822346249854]
  This study presents a framework whereby several empathetic chatbots are based on understanding users' implied feelings and replying empathetically for multiple dialogue turns. We call these chatbots CheerBots. CheerBots can be retrieval-based or generative-based and were finetuned by deep reinforcement learning. To respond in an empathetic way, we develop a simulating agent, a Conceptual Human Model, as aids for CheerBots in training with considerations on changes in user's emotional states in the future to arouse sympathy.
  arXiv  Detail & Related papers  (2021-10-08T07:44:47Z)
• Detecting Bot-Generated Text by Characterizing Linguistic Accommodation in Human-Bot Interactions [9.578008322407928]
  Language generation models' democratization makes it easier to generate human-like text at-scale for nefarious activities. It is essential to understand how people interact with bots and develop methods to detect bot-generated text. This paper shows that bot-generated text detection methods are more robust across datasets and models if we use information about how people respond to it.
  arXiv  Detail & Related papers  (2021-06-02T14:10:28Z)
• Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention [55.77218465471519]
  This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework included a guiding robot and an interlocutor model that plays the role of humans. We examined our framework using three experimental setups and evaluate the guiding robot with four different metrics to demonstrated flexibility and performance advantages.
  arXiv  Detail & Related papers  (2021-03-30T15:24:37Z)
• Detection of Novel Social Bots by Ensembles of Specialized Classifiers [60.63582690037839]
  Malicious actors create inauthentic social media accounts controlled in part by algorithms, known as social bots, to disseminate misinformation and agitate online discussion. We show that different types of bots are characterized by different behavioral features. We propose a new supervised learning method that trains classifiers specialized for each class of bots and combines their decisions through the maximum rule.
  arXiv  Detail & Related papers  (2020-06-11T22:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.