Exploring the Use of Abusive Generative AI Models on Civitai

• URL: http://arxiv.org/abs/2407.12876v2
• Date: Sat, 20 Jul 2024 19:59:42 GMT
• Title: Exploring the Use of Abusive Generative AI Models on Civitai
• Authors: Yiluo Wei, Yiming Zhu, Pan Hui, Gareth Tyson,
• Abstract summary: We study the use of Civitai, the largest AIGC social platform, for generating abusive content. We construct a comprehensive dataset covering 87K models and 2M images. We discuss strategies for moderation to better govern these platforms.
• Score: 22.509955105958625
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The rise of generative AI is transforming the landscape of digital imagery, and exerting a significant influence on online creative communities. This has led to the emergence of AI-Generated Content (AIGC) social platforms, such as Civitai. These distinctive social platforms allow users to build and share their own generative AI models, thereby enhancing the potential for more diverse artistic expression. Designed in the vein of social networks, they also provide artists with the means to showcase their creations (generated from the models), engage in discussions, and obtain feedback, thus nurturing a sense of community. Yet, this openness also raises concerns about the abuse of such platforms, e.g., using models to disseminate deceptive deepfakes or infringe upon copyrights. To explore this, we conduct the first comprehensive empirical study of an AIGC social platform, focusing on its use for generating abusive content. As an exemplar, we construct a comprehensive dataset covering Civitai, the largest available AIGC social platform. Based on this dataset of 87K models and 2M images, we explore the characteristics of content and discuss strategies for moderation to better govern these platforms.

Related papers

• Social Reward: Evaluating and Enhancing Generative AI through Million-User Feedback from an Online Creative Community [63.949893724058846]
  Social reward as a form of community recognition provides a strong source of motivation for users of online platforms to engage and contribute with content. This work pioneers a paradigm shift, unveiling Social Reward - an innovative reward modeling framework. We embark on an extensive journey of dataset curation and refinement, drawing from Picsart: an online visual creation and editing platform.
  arXiv  Detail & Related papers  (2024-02-15T10:56:31Z)
• State of the Art on Diffusion Models for Visual Computing [191.6168813012954]
  This report introduces the basic mathematical concepts of diffusion models, implementation details and design choices of the popular Stable Diffusion model. We also give a comprehensive overview of the rapidly growing literature on diffusion-based generation and editing. We discuss available datasets, metrics, open challenges, and social implications.
  arXiv  Detail & Related papers  (2023-10-11T05:32:29Z)
• AI-Generated Images as Data Source: The Dawn of Synthetic Era [61.879821573066216]
  generative AI has unlocked the potential to create synthetic images that closely resemble real-world photographs. This paper explores the innovative concept of harnessing these AI-generated images as new data sources. In contrast to real data, AI-generated data exhibit remarkable advantages, including unmatched abundance and scalability.
  arXiv  Detail & Related papers  (2023-10-03T06:55:19Z)
• Understanding Place Identity with Generative AI [2.1441748927508506]
  generative AI models have the potential to capture the collective image of cities that can make them distinguishable. This study is among the first attempts to explore the capabilities of generative AI in understanding human perceptions of the built environment.
  arXiv  Detail & Related papers  (2023-06-07T02:32:45Z)
• AI Imagery and the Overton Window [0.0]
  This paper is a literature review examining the concerns facing both AI developers and users today. It discusses legalization challenges and ethical concerns, and concludes with how AI generative models can be tremendously useful.
  arXiv  Detail & Related papers  (2023-05-31T18:01:04Z)
• Guiding AI-Generated Digital Content with Wireless Perception [69.51950037942518]
  We introduce an integration of wireless perception with AI-generated content (AIGC) to improve the quality of digital content production. The framework employs a novel multi-scale perception technology to read user's posture, which is difficult to describe accurately in words, and transmits it to the AIGC model as skeleton images. Since the production process imposes the user's posture as a constraint on the AIGC model, it makes the generated content more aligned with the user's requirements.
  arXiv  Detail & Related papers  (2023-03-26T04:39:03Z)
• A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT [63.58711128819828]
  ChatGPT and other Generative AI (GAI) techniques belong to the category of Artificial Intelligence Generated Content (AIGC) The goal of AIGC is to make the content creation process more efficient and accessible, allowing for the production of high-quality content at a faster pace.
  arXiv  Detail & Related papers  (2023-03-07T20:36:13Z)
• SoMin.ai: Personality-Driven Content Generation Platform [60.49416044866648]
  We showcase the World's first personality-driven marketing content generation platform, called SoMin.ai. The platform combines deep multi-view personality profiling framework and style generative adversarial networks. It can be used for the enhancement of the social networking user experience as well as for content marketing routines.
  arXiv  Detail & Related papers  (2020-11-30T08:33:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.