As AI becomes more commonplace, bad actors are inevitably using these tools for nefarious purposes. Some people are using the best AI chatbots to sell AI-generated nudes of celebrities or to create soulless George Carlin specials. However, a recent report paints a very dystopian picture.
Meta, the company that owns Instagram, Facebook, WhatsApp, and a popular line of VR headsets, is at the center of a new wave of perverts creating AI-generated content of kids. Perverts are flocking to Instagram and similar platforms to find this material.
While most people are concerned with turning off Meta AI on Facebook and Instagram, many are unaware of the disgusting ways people are using AI. A recent Forbes report suggests that Meta isn't doing enough to stop users from flooding its social media with disturbing generated imagery of children.
The report details the types of users commenting on these posts, which primarily affect Meta's Instagram and the popular TikTok app. National agencies have struggled to combat Child Sexual Abuse Material (CSAM) in the past, and AI makes it even easier to create such content, despite it being illegal even if it’s fake.
This disturbing trend shouldn't be surprising. A December 2023 study (via Futurism) revealed that some AI tools, such as Stable Diffusion, used CSAM content in their datasets. Even Google's AI overview admits to using CSAM content. Unfortunately, this allows people to use the same dataset to create child abuse material.
Even worse, some content found on Instagram and TikTok during the Forbes investigation is technically legal, since the individuals in the images are fake and not nude. However, the images are still sexually suggestive, and tech companies should remove them because they attract a harmful community. Some comments read, "I would like to have a pretty little virgin like that in my hands to make it mine," or "She’s not real, but I’ve got a great imagination."
Both TikTok and Meta reportedly deleted the accounts and posts referenced in the Forbes article, stating that the content violates their social media policies.
While this case is particularly disturbing, it shouldn't be a huge shock. A Copilot engineer warned the FTC about AI creating harmful content, and Copilot is owned by one of the biggest tech companies in the world, Microsoft. As AI generation tools become more accessible, like ChatGPT 4o being free for any user, more cases like this will arise. Laws need to be put in place to protect vulnerable individuals.
We've reached out to Meta and TikTok's PR for comment.