A Marketplace for AI-Generated Adult Content and Deepfakes

• URL: http://arxiv.org/abs/2601.09117v2
• Date: Wed, 21 Jan 2026 02:44:40 GMT
• Title: A Marketplace for AI-Generated Adult Content and Deepfakes
• Abstract summary: Civitai, a popular community-driven platform for AI-generated content, operates a monetized feature called Bounties.<n>We conduct a longitudinal analysis of all publicly available bounty requests collected over a 14-month period following the platform's launch.<n>We find that the bounty marketplace is dominated by tools that let users steer AI models toward content they were not trained to generate.
• Score: 2.1392057118867123
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI systems increasingly enable the production of highly realistic synthetic media. Civitai, a popular community-driven platform for AI-generated content, operates a monetized feature called Bounties, which allows users to commission the generation of content in exchange for payment. To examine how this mechanism is used and what content it incentivizes, we conduct a longitudinal analysis of all publicly available bounty requests collected over a 14-month period following the platform's launch. We find that the bounty marketplace is dominated by tools that let users steer AI models toward content they were not trained to generate. At the same time, requests for content that is "Not Safe For Work" are widespread and have increased steadily over time, now comprising a majority of all bounties. Participation in bounty creation is uneven, with 20% of requesters accounting for roughly half of requests. Requests for "deepfake" - media depicting identifiable real individuals - exhibit a higher concentration than other types of bounties. A nontrivial subset of these requests involves explicit deepfakes despite platform policies prohibiting such content. These bounties disproportionately target female celebrities, revealing a pronounced gender asymmetry in social harm. Together, these findings show how monetized, community-driven generative AI platforms can produce gendered harms, raising questions about consent, governance, and enforcement.

Related papers

• Can Media Act as a Soft Regulator of Safe AI Development? A Game Theoretical Analysis [57.68073583427415]
  We study whether media coverage has the potential to push AI creators into the production of safe products.<n>Our results reveal that media is indeed able to foster cooperation between creators and users, but not always.<n>By shaping public perception and holding developers accountable, media emerges as a powerful soft regulator.
  arXiv  Detail & Related papers  (2025-09-02T12:13:34Z)
• A New Incentive Model For Content Trust [0.0]
  This paper outlines an incentive-driven and decentralized approach to verifying the veracity of digital content at scale.<n>We believe that it could be possible to foster a self-propelling paradigm shift to combat misinformation through a community-based governance model.
  arXiv  Detail & Related papers  (2025-07-14T06:41:55Z)
• A Comprehensive Content Verification System for ensuring Digital Integrity in the Age of Deep Fakes [0.0]
  This paper discusses a solution, a Content Verification System, designed to authenticate images and videos shared as posts or stories across the digital landscape.<n>Going beyond the limitations of blue ticks, this system empowers individuals and influencers to validate the authenticity of their digital footprint, safeguarding their reputation in an interconnected world.
  arXiv  Detail & Related papers  (2024-11-29T14:47:47Z)
• Social Media Bot Policies: Evaluating Passive and Active Enforcement [0.0]
  Multimodal Foundation Models (MFMs) may facilitate malicious actors in the exploitation of online users. We examined the bot and content policies of eight popular social media platforms: X (formerly Twitter), Instagram, Facebook, Threads, TikTok, Mastodon, Reddit, and LinkedIn. Our findings indicate significant vulnerabilities within the current enforcement mechanisms of these platforms.
  arXiv  Detail & Related papers  (2024-09-27T17:28:25Z)
• English offensive text detection using CNN based Bi-GRU model [1.102674168371806]
  Social networking sites like Facebook and Twitter provide platforms to share vast amounts of content with a single click. These platforms do not impose restrictions on the uploaded content, which may include abusive language and explicit images unsuitable for social media. We propose a new Bi-GRU-CNN model to classify whether the text is offensive or not.
  arXiv  Detail & Related papers  (2024-09-24T01:29:24Z)
• "Violation of my body:" Perceptions of AI-generated non-consensual (intimate) imagery [22.68931586977199]
  AI technology has enabled the creation of deepfakes: hyper-realistic synthetic media. We surveyed 315 individuals in the U.S. on their views regarding the hypothetical non-consensual creation of deepfakes depicting them.
  arXiv  Detail & Related papers  (2024-06-08T16:57:20Z)
• An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software [64.367830425115]
  Social media platforms are being increasingly misused to spread toxic content, including hate speech, malicious advertising, and pornography. Despite tremendous efforts in developing and deploying content moderation methods, malicious users can evade moderation by embedding texts into images. We propose a metamorphic testing framework for content moderation software.
  arXiv  Detail & Related papers  (2023-08-18T20:33:06Z)
• Modeling Content Creator Incentives on Algorithm-Curated Platforms [76.53541575455978]
  We study how algorithmic choices affect the existence and character of (Nash) equilibria in exposure games. We propose tools for numerically finding equilibria in exposure games, and illustrate results of an audit on the MovieLens and LastFM datasets.
  arXiv  Detail & Related papers  (2022-06-27T08:16:59Z)
• Manipulating Twitter Through Deletions [64.33261764633504]
  Research into influence campaigns on Twitter has mostly relied on identifying malicious activities from tweets obtained via public APIs. Here, we provide the first exhaustive, large-scale analysis of anomalous deletion patterns involving more than a billion deletions by over 11 million accounts. We find that a small fraction of accounts delete a large number of tweets daily. First, limits on tweet volume are circumvented, allowing certain accounts to flood the network with over 26 thousand daily tweets. Second, coordinated networks of accounts engage in repetitive likes and unlikes of content that is eventually deleted, which can manipulate ranking algorithms.
  arXiv  Detail & Related papers  (2022-03-25T20:07:08Z)
• News consumption and social media regulations policy [70.31753171707005]
  We analyze two social media that enforced opposite moderation methods, Twitter and Gab, to assess the interplay between news consumption and content regulation. Our results show that the presence of moderation pursued by Twitter produces a significant reduction of questionable content. The lack of clear regulation on Gab results in the tendency of the user to engage with both types of content, showing a slight preference for the questionable ones which may account for a dissing/endorsement behavior.
  arXiv  Detail & Related papers  (2021-06-07T19:26:32Z)
• SoMin.ai: Personality-Driven Content Generation Platform [60.49416044866648]
  We showcase the World's first personality-driven marketing content generation platform, called SoMin.ai. The platform combines deep multi-view personality profiling framework and style generative adversarial networks. It can be used for the enhancement of the social networking user experience as well as for content marketing routines.
  arXiv  Detail & Related papers  (2020-11-30T08:33:39Z)
• Quantifying the Vulnerabilities of the Online Public Square to Adversarial Manipulation Tactics [43.98568073610101]
  We use a social media model to quantify the impacts of several adversarial manipulation tactics on the quality of content. We find that the presence of influential accounts, a hallmark of social media, exacerbates the vulnerabilities of online communities to manipulation. These insights suggest countermeasures that platforms could employ to increase the resilience of social media users to manipulation.
  arXiv  Detail & Related papers  (2019-07-13T21:12:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.