Real Women's Deepfake Images Are Being Sold on Civitai Marketplace

Deepfake Image Crisis in the AI Marketplace

A study by Stanford and Indiana University researchers, not yet peer-reviewed, has revealed alarming trends on the online marketplace Civitai, where AI-generated content is bought and sold. It was determined that deepfake images of real people, particularly women, are being requested and sold on the platform.

The Study's Striking Findings

Researchers examined "bounty" requests on Civitai between mid-2023 and the end of 2024. During this period, it was found that 90% of deepfake requests on the platform targeted women. The requests included public figures like influencer Charli D'Amelio and singer Gracie Abrams, as well as women active in specific niches, such as artists who record ASMR videos.

Even more concerning was a user requesting a deepfake of a woman claimed to be his wife. Users provide links to social media profiles to enable scraping images from the web, requesting models to generate the person's entire body, accurately capture their tattoos, or allow for hair color changes.

LoRA Files and the Platform's Role

Civitai not only facilitates the buying and selling of images and videos but also instruction files called LoRAs, which steer mainstream AI models like Stable Diffusion to generate content they were not trained on. According to the research, 86% of deepfake requests on Civitai were for these LoRA files.

The platform also provides educational resources on how to use external tools to further customize the outputs of image generators. The site hosts user posts containing details on how to steer models to produce pornography. Researchers note that the amount of pornographic content on the platform has increased, with the majority of weekly requests now being for NSFW (not safe for work) content.

Moderation and Legal Liability Questions

Civitai automatically tags bounty requests for deepfakes and lists a process where the person featured in the content can manually request removal. However, this system outsources moderation to the general public and prevents the platform from taking proactive action.

Professor Ryan Calo, a technology and AI expert at the University of Washington School of Law, states that the legal liability of tech companies for their users' actions is not entirely clear. Calo emphasizes that existing legal protections are not unlimited, saying, "You can't knowingly facilitate illegal transactions on your website."

The Platform's Response and Funding Reality

Civitai announced in May 2025 that it would ban all deepfake content. However, according to MIT Technology Review's verification, numerous deepfake requests posted before this ban remain active on the platform, and many of the winning submissions fulfilling these requests are ready for purchase.

The platform received a $5 million investment from venture capital firm Andreessen Horowitz (a16z) in November 2023. Civitai's co-founder and CEO Justin Maier spoke about the goal of building the main place where people find and share AI models for their individual purposes.

However, researchers and experts emphasize that adult deepfakes do not receive the same level of attention from content platforms or the venture capital firms funding them. Professor Calo assesses, "They are not scared enough of this. They are overly tolerant of it."

These developments reignite debates about the ethical use of AI technologies and the need for regulation. Similarly, the responsibilities of AI companies regarding copyright and user data are increasingly being questioned.