The Looming Threat of Fake and LLM-generated LinkedIn Profiles: Challenges and Opportunities for Detection and Prevention

• URL: http://arxiv.org/abs/2307.11864v1
• Date: Fri, 21 Jul 2023 19:09:24 GMT
• Title: The Looming Threat of Fake and LLM-generated LinkedIn Profiles: Challenges and Opportunities for Detection and Prevention
• Authors: Navid Ayoobi, Sadat Shahriar, Arjun Mukherjee
• Abstract summary: We present a novel method for detecting fake and Large Language Model (LLM)-generated profiles in the LinkedIn Online Social Network. We show that the suggested method can distinguish between legitimate and fake profiles with an accuracy of about 95% across all word embeddings.
• Score: 0.8808993671472349
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present a novel method for detecting fake and Large Language Model (LLM)-generated profiles in the LinkedIn Online Social Network immediately upon registration and before establishing connections. Early fake profile identification is crucial to maintaining the platform's integrity since it prevents imposters from acquiring the private and sensitive information of legitimate users and from gaining an opportunity to increase their credibility for future phishing and scamming activities. This work uses textual information provided in LinkedIn profiles and introduces the Section and Subsection Tag Embedding (SSTE) method to enhance the discriminative characteristics of these data for distinguishing between legitimate profiles and those created by imposters manually or by using an LLM. Additionally, the dearth of a large publicly available LinkedIn dataset motivated us to collect 3600 LinkedIn profiles for our research. We will release our dataset publicly for research purposes. This is, to the best of our knowledge, the first large publicly available LinkedIn dataset for fake LinkedIn account detection. Within our paradigm, we assess static and contextualized word embeddings, including GloVe, Flair, BERT, and RoBERTa. We show that the suggested method can distinguish between legitimate and fake profiles with an accuracy of about 95% across all word embeddings. In addition, we show that SSTE has a promising accuracy for identifying LLM-generated profiles, despite the fact that no LLM-generated profiles were employed during the training phase, and can achieve an accuracy of approximately 90% when only 20 LLM-generated profiles are added to the training set. It is a significant finding since the proliferation of several LLMs in the near future makes it extremely challenging to design a single system that can identify profiles created with various LLMs.

Related papers

• Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization [33.513689684998035]
  The concept of persona, originally adopted in dialogue literature, has re-surged as a promising framework for tailoring large language models to specific context. To close the gap, we present a comprehensive survey to categorize the current state of the field.
  arXiv  Detail & Related papers  (2024-06-03T10:08:23Z)
• Are you still on track!? Catching LLM Task Drift with Activations [55.75645403965326]
  Large Language Models (LLMs) are routinely used in retrieval-augmented applications to orchestrate tasks and process inputs from users and other sources. This opens the door to prompt injection attacks, where the LLM receives and acts upon instructions from supposedly data-only sources, thus deviating from the user's original instructions. We define this as task drift, and we propose to catch it by scanning and analyzing the LLM's activations. We show that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions, without being trained on any of these attacks.
  arXiv  Detail & Related papers  (2024-06-02T16:53:21Z)
• SPOT: Text Source Prediction from Originality Score Thresholding [6.790905400046194]
  countermeasures aim at detecting misinformation, usually involve domain specific models trained to recognize the relevance of any information. Instead of evaluating the validity of the information, we propose to investigate LLM generated text from the perspective of trust.
  arXiv  Detail & Related papers  (2024-05-30T21:51:01Z)
• ReMoDetect: Reward Models Recognize Aligned LLM's Generations [55.06804460642062]
  Large language models (LLMs) generate human-preferable texts. We propose two training schemes to further improve the detection ability of the reward model.
  arXiv  Detail & Related papers  (2024-05-27T17:38:33Z)
• CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models [60.59638232596912]
  We introduce CLAMBER, a benchmark for evaluating large language models (LLMs) Building upon the taxonomy, we construct 12K high-quality data to assess the strengths, weaknesses, and potential risks of various off-the-shelf LLMs. Our findings indicate the limited practical utility of current LLMs in identifying and clarifying ambiguous user queries.
  arXiv  Detail & Related papers  (2024-05-20T14:34:01Z)
• Ghost Sentence: A Tool for Everyday Users to Copyright Data from Large Language Models [55.321010757641524]
  Web user data plays a central role in the ecosystem of pre-trained large language models (LLMs) In this work, we suggest that users repeatedly insert personal passphrases into their documents. Once they are identified in the generated content of LLMs, users can be sure that their data is used for training.
  arXiv  Detail & Related papers  (2024-03-23T06:36:32Z)
• TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification [41.25887364156612]
  We describe the novel fingerprinting problem of Black-box Identity Verification (BBIV) The goal is to determine whether a third-party application uses a certain LLM through its chat function. We propose a method called Targeted Random Adversarial Prompt (TRAP) that identifies the specific LLM in use.
  arXiv  Detail & Related papers  (2024-02-20T13:20:39Z)
• Do Membership Inference Attacks Work on Large Language Models? [145.90022632726883]
  Membership inference attacks (MIAs) attempt to predict whether a particular datapoint is a member of a target model's training data. We perform a large-scale evaluation of MIAs over a suite of language models trained on the Pile, ranging from 160M to 12B parameters. We find that MIAs barely outperform random guessing for most settings across varying LLM sizes and domains.
  arXiv  Detail & Related papers  (2024-02-12T17:52:05Z)
• Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models [52.98743860365194]
  We propose a new fine-tuning method called Self-Play fIne-tuNing (SPIN) At the heart of SPIN lies a self-play mechanism, where the LLM refines its capability by playing against instances of itself. This sheds light on the promise of self-play, enabling the achievement of human-level performance in LLMs without the need for expert opponents.
  arXiv  Detail & Related papers  (2024-01-02T18:53:13Z)
• Beyond Memorization: Violating Privacy Via Inference with Large Language Models [2.9373912230684565]
  We present the first comprehensive study on the capabilities of pretrained language models to infer personal attributes from text. Our findings highlight that current LLMs can infer personal data at a previously unattainable scale.
  arXiv  Detail & Related papers  (2023-10-11T08:32:46Z)
• Identifying Fake Profiles in LinkedIn [0.22843885788439797]
  We identify the minimal set of profile data necessary for identifying fake profiles in LinkedIn. We propose an appropriate data mining approach for fake profile identification. Our approach can identify fake profiles with 87% accuracy and 94% True Negative Rate.
  arXiv  Detail & Related papers  (2020-06-02T04:15:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.