On pitfalls (and advantages) of sophisticated large language models

• URL: http://arxiv.org/abs/2303.17511v1
• Date: Sat, 25 Feb 2023 11:14:39 GMT
• Title: On pitfalls (and advantages) of sophisticated large language models
• Abstract summary: Overreliance on large language models can have disruptive consequences. Privacy rights, the possibility of circulating counterfeits of humans, and, last but not least, it makes a massive spread of misinformation possible.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Natural language processing based on large language models (LLMs) is a booming field of AI research. After neural networks have proven to outperform humans in games and practical domains based on pattern recognition, we might stand now at a road junction where artificial entities might eventually enter the realm of human communication. However, this comes with serious risks. Due to the inherent limitations regarding the reliability of neural networks, overreliance on LLMs can have disruptive consequences. Since it will be increasingly difficult to distinguish between human-written and machine-generated text, one is confronted with new ethical challenges. This begins with the no longer undoubtedly verifiable human authorship and continues with various types of fraud, such as a new form of plagiarism. This also concerns the violation of privacy rights, the possibility of circulating counterfeits of humans, and, last but not least, it makes a massive spread of misinformation possible.

Related papers

• Are they human? Detecting large language models by probing human memory constraints [7.063564727972718]
  The validity of online behavioral research relies on study participants being human rather than machine.<n>General-purpose agents based on large language models (LLMs) can now solve many challenges, threatening the validity of online behavioral research.<n>We show that cognitive modeling on a standard serial recall task can be used to distinguish online participants from LLMs.
  arXiv  Detail & Related papers  (2026-03-10T21:15:19Z)
• LVLMs and Humans Ground Differently in Referential Communication [33.82075906105276]
  We present a factorial design involving director-matcher pairs that interact with multiple turns in repeated rounds to match pictures of objects not associated with any obvious lexicalized labels.<n>We release the online pipeline for data collection, the tools and analyses for accuracy, efficiency, and lexical overlap, and a corpus of 356 dialogues that unmasks LVLMs' limitations in interactively resolving referring expressions.
  arXiv  Detail & Related papers  (2026-01-27T16:52:20Z)
• Uncertainty in Authorship: Why Perfect AI Detection Is Mathematically Impossible [0.0]
  We argue that there is a fundamental trade-off: the more confidently one tries to identify whether a text was written by a human or an AI, the more one risks disrupting the text's natural flow and authenticity.<n>Our analysis shows that when AI-generated text closely mimics human writing, perfect detection becomes not just technologically difficult but theoretically impossible.
  arXiv  Detail & Related papers  (2025-09-15T13:33:32Z)
• Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing [66.04926909181653]
  We argue for anthropomimetic uncertainty, meaning that intuitive and trustworthy uncertainty communication requires a degree of linguistic authenticity and personalization to the user.<n>We conclude by pointing out unique factors in human-machine communication of uncertainty and deconstruct the data biases that influence machine uncertainty communication.
  arXiv  Detail & Related papers  (2025-07-11T14:07:22Z)
• When Detection Fails: The Power of Fine-Tuned Models to Generate Human-Like Social Media Text [13.14749943120523]
  Social media represents a significant attack vector in online influence campaigns.<n>We create a dataset of 505,159 AI-generated social media posts from a combination of open-source, closed-source, and fine-tuned LLMs.<n>We show that while the posts can be detected under typical research assumptions, under the more realistic assumption that an attacker will not release their fine-tuned model to the public, detectability drops dramatically.
  arXiv  Detail & Related papers  (2025-06-11T17:51:28Z)
• SLURG: Investigating the Feasibility of Generating Synthetic Online Fallacious Discourse [0.0]
  We explore the definition, and extrapolation of fallacies as they pertain to the automatic detection of manipulation on social media. In particular we explore how these logical fallacies might appear in the real world i.e internet forums. We discovered a prevalence of misinformation / misguided intention in discussion boards specifically centered around the Ukrainian Russian Conflict.
  arXiv  Detail & Related papers  (2025-04-16T20:03:47Z)
• Strong and weak alignment of large language models with human values [1.6590638305972631]
  Minimizing negative impacts of Artificial Intelligent (AI) systems requires them to be able to align with human values. We argue that this is required for AI systems like large language models (LLMs) to be able to recognize situations presenting a risk that human values may be flouted. We propose a new thought experiment that we call "the Chinese room with a word transition dictionary", in extension of John Searle's famous proposal.
  arXiv  Detail & Related papers  (2024-08-05T11:27:51Z)
• Theory of Mind abilities of Large Language Models in Human-Robot Interaction : An Illusion? [18.770522926093786]
  Large Language Models have shown exceptional generative abilities in various natural language and generation tasks. We study a special application of ToM abilities that has higher stakes and possibly irreversible consequences. We focus on the task of Perceived Behavior Recognition, where a robot employs a Large Language Model (LLM) to assess the robot's generated behavior in a manner similar to human observer.
  arXiv  Detail & Related papers  (2024-01-10T18:09:36Z)
• Towards Possibilities & Impossibilities of AI-generated Text Detection: A Survey [97.33926242130732]
  Large Language Models (LLMs) have revolutionized the domain of natural language processing (NLP) with remarkable capabilities of generating human-like text responses. Despite these advancements, several works in the existing literature have raised serious concerns about the potential misuse of LLMs. To address these concerns, a consensus among the research community is to develop algorithmic solutions to detect AI-generated text.
  arXiv  Detail & Related papers  (2023-10-23T18:11:32Z)
• Amplifying Limitations, Harms and Risks of Large Language Models [1.0152838128195467]
  We present this article as a small gesture in an attempt to counter what appears to be exponentially growing hype around Artificial Intelligence. It may also help those outside of the field to become more informed about some of the limitations of AI technology.
  arXiv  Detail & Related papers  (2023-07-06T11:53:45Z)
• Can AI-Generated Text be Reliably Detected? [54.670136179857344]
  Unregulated use of LLMs can potentially lead to malicious consequences such as plagiarism, generating fake news, spamming, etc. Recent works attempt to tackle this problem either using certain model signatures present in the generated text outputs or by applying watermarking techniques. In this paper, we show that these detectors are not reliable in practical scenarios.
  arXiv  Detail & Related papers  (2023-03-17T17:53:19Z)
• Is This Abstract Generated by AI? A Research for the Gap between AI-generated Scientific Text and Human-written Scientific Text [13.438933219811188]
  We investigate the gap between scientific content generated by AI and written by humans. We find that there exists a writing style'' gap between AI-generated scientific text and human-written scientific text.
  arXiv  Detail & Related papers  (2023-01-24T04:23:20Z)
• When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment [96.77970239683475]
  AI systems need to be able to understand, interpret and predict human moral judgments and decisions. A central challenge for AI safety is capturing the flexibility of the human moral mind. We present a novel challenge set consisting of rule-breaking question answering.
  arXiv  Detail & Related papers  (2022-10-04T09:04:27Z)
• Human Heuristics for AI-Generated Language Are Flawed [8.465228064780744]
  We study whether verbal self-presentations, one of the most personal and consequential forms of language, were generated by AI. We experimentally demonstrate that these wordings make human judgment of AI-generated language predictable and manipulable. We discuss solutions, such as AI accents, to reduce the deceptive potential of language generated by AI.
  arXiv  Detail & Related papers  (2022-06-15T03:18:56Z)
• Searching for the Essence of Adversarial Perturbations [73.96215665913797]
  We show that adversarial perturbations contain human-recognizable information, which is the key conspirator responsible for a neural network's erroneous prediction. This concept of human-recognizable information allows us to explain key features related to adversarial perturbations.
  arXiv  Detail & Related papers  (2022-05-30T18:04:57Z)
• Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting [135.0863818867184]
  artificial neural variability (ANV) helps artificial neural networks learn some advantages from natural'' neural networks. ANV plays as an implicit regularizer of the mutual information between the training data and the learned model. It can effectively relieve overfitting, label noise memorization, and catastrophic forgetting at negligible costs.
  arXiv  Detail & Related papers  (2020-11-12T06:06:33Z)
• Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans [75.15855405318855]
  We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing. Although the network was solely trained to predict the next word in a large corpus, analysis showed the emergence of specialized units that successfully handled local and long-distance syntactic agreement. We tested the model's predictions in a behavioral experiment where humans detected violations in number agreement in sentences with systematic variations in the singular/plural status of multiple nouns.
  arXiv  Detail & Related papers  (2020-06-19T12:00:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.