Most social users will have come across an influencer who looks a little… off.
Maybe their facial features are a bit too symmetrical, and their poses are a bit too rigid. Chances are, you’re not looking at a human at all – but an AI-generated forgery.
In some cases, these AI influencers are reasonably benign – just digital forms of their real counterparts but not overtly trying to deceive or manipulate.
However, this isn’t always the case. Disturbingly, there’s a network of Instagram accounts using artificial intelligence to create fake influencers with Down syndrome.
These bad actors steal content from real creators, then leverage AI to swap in computer-generated faces of people with Down syndrome. The goal? To exploit a vulnerable community for likes, shares, and ultimately, cash.
But the deception doesn’t end there. Many of these accounts link out to shady adult websites, where the AI-generated content is monetized.
Sadly, this is just the latest evolution of the “AI pimping” trend, where unscrupulous operators use machine learning to create counterfeit influencers for monetary gain. It’s not just Down syndrome, but fake amputee models, burn victims, and other forms of AI-generated pornography.
AI image and video models are now approaching a level of realism that makes them very viable substitutes for real humans. It’s affecting the fashion industry – where real models are facing replacement at the hands of the AI clones.
Even household names like H&M are wading into these murky waters. The fast fashion giant recently announced a campaign featuring AI-generated “digital twins” of real models. Back in 2023, a company called lalaland.ai released tools for creating AI models for a subscription fee.
While H&M insists the models maintain control over their digital likenesses, many in the industry are skeptical. After all, in an era of cost-cutting and consolidation, why hire human talent when you can license a cheap, infinitely replicable digital avatar?
The latest, most insidious twist here concerns the fundamental dignity and humanity of marginalized communities.
People with Down syndrome – or any disability – are not props to be manipulated for profit.
Moreover, the proliferation of AI-generated content threatens to erode public trust in media altogether. If we cannot trust the images we see online, the very foundation of digital discourse starts to erode.
So next time you’re scrolling through your feed and an influencer seems too good to be true, trust your gut.
