TikTok recently launched a new AI tool that allows users to create digital avatars to say whatever they want. The tool uses paid actors to generate content based on user input, and thanks to TikTok's tight control, it typically avoids anything controversial. Unfortunately, the internal version was accidentally shared with the public.
Much like the concepts behind the best AI image generators and video generators like Luma Dream Machine or the upcoming OpenAI Sora, TikTok’s Symphony lets businesses create advertisements using paid actors, allowing companies to promote their products with seemingly real people. Normally, this content is restricted to adhere to TikTok's guidelines.
However, a mistake has given users malicious control over the Symphony tool. As reported by CNN, TikTok's AI Avatars have been misused to disseminate excerpts from Hitler's Mein Kampf, Osama Bin Laden's letter to America, and to suggest drinking bleach, among other problematic examples.
Unsurprisingly, this isn't the first instance in recent months where users have successfully given harmful prompts to AI. Google's AI overviews have advised people to drink urine to treat kidney stones, Copilot has been accused of generating harmful imagery, and on the flip side, Microsoft recently discontinued its privacy-invading Recall AI tool. It seems that, whenever general audiences are giving access to AI, some will inevitably use the AI for negative reasons.
TikTok has since issued a statement to CNN about the issue, calling it a "technical error" and clarifying that the Symphony version shared was an "internal testing model" likely lacking necessary safety features. Since the CNN report, TikTok has removed the internal version of the model, while the standard version remains available.
While some, like the so-called "Godfather of AI," believe the technology could lead to human extinction, it seems more likely that people will misuse AI tools to harm each other. In any case, it's concerning how frequently AI tools can malfunction and be misused.