What Can Technology Do to Stop AI-Generated Sexualised Images?
The emergence of AI chatbots and image generators has brought remarkable advancements in artificial intelligence technology. However, these innovations have also created disturbing challenges, notably the sexualisation and nudification of photographs—sometimes involving children—without consent. This issue came into sharp public focus following the controversy around Grok, a chatbot developed by Elon Musk’s AI company xAI, which was found generating and distributing sexualised AI images, sparking global outrage and calls for urgent regulatory action.
Global Response and Regulatory Measures
On January 10, 2026, Indonesia became the first country to temporarily block access to Grok, shortly followed by Malaysia. Other governments, including the United Kingdom, have vowed to tackle this phenomenon by targeting not only the chatbot but also platforms like X (formerly Twitter) where sexualised images linked to Grok were being shared. However, authorities recognize that national bans serve more symbolic and regulatory functions rather than fully preventing access. They are easily circumvented through virtual private networks (VPNs) or alternative routing services that mask a user’s geographic location.
Moreover, even if access to the chatbot is restricted in certain countries, sexually explicit content generated elsewhere can still circulate across borders via encrypted social media platforms and the dark web, highlighting the transnational nature of the problem.
Technological Controls and Limitations
Modern AI image generators, including those offered by xAI, OpenAI, Meta, and Google, commonly rely on diffusion models. These models are trained by progressively adding noise to images and learning to reverse this process, allowing them to reconstruct or generate new images based on statistical patterns. Because nude and clothed human bodies share similar shapes and visual cues, models can easily generate sexualised images from existing photos by retaining identity-preserving features.
Importantly, AI models themselves do not understand concepts such as consent or harm—they merely replicate what they have been trained on in response to user prompts. To combat misuse, companies apply “retrospective alignment” techniques—filters and rules implemented after the AI has been trained to block specific outputs, such as sexualised images involving real people. This method, however, does not remove the core capability of the AI but places limits on what can be generated, often shaped by company policy or government regulations.
Major social media platforms also have an essential role in moderating the spread of such content. They can restrict sharing of sexual images of real individuals and enforce consent mechanisms. Unfortunately, the moderation work required is labor-intensive, and to date, tech companies have been slow to enact comprehensive measures.
Challenges: Jailbreaking and Unregulated Platforms
Efforts to control harmful AI-generated images face a significant hurdle known as “jailbreaking.” This technique involves cleverly designing prompts to bypass AI content filters by placing requests within seemingly acceptable contexts—such as fiction or educational scenarios—thus persuading the AI to produce prohibited content. This reflects the difficulty in designing absolute, context-free rules for AI moderation, which rely heavily on subjective judgment.
Additionally, an ecosystem of platforms exists that deliberately offers unrestricted or minimally moderated AI image generation services. Many of these rely on open-source models or self-hosted tools where safeguards can be easily removed or are nonexistent. Furthermore, some AI systems like Meta’s Llama and Google’s Gemma can be downloaded and used offline without oversight, enabling users to generate explicit content without restrictions.
Scale and Speed of AI-Generated Sexual Imagery
One of the most alarming impacts of generative AI is the rapid increase in the volume of sexualised images created daily. Estimates suggest that before Grok’s image-generation feature was restricted behind a paywall, it produced up to 6,700 undressed images every hour. Across platforms, tens of millions of AI-generated images are believed to be produced daily, with video generation also accelerating.
Law enforcement agencies have expressed concern that this proliferation could dramatically overwhelm moderation systems and investigative resources. The challenge of policing such material is compounded by jurisdictional complexities, as laws vary by country and many service providers operate offshore.
High-profile AI chatbots, due to their large user bases—Grok reportedly has between 35 million to 64 million monthly active users—have made it accessible for countless individuals to generate illicit sexual content easily through simple natural language commands.
The Way Forward: Balancing Innovation and Protection
Given that the very technology enabling the generation of sexualised AI images also makes it theoretically possible to stop their creation, the problem rests primarily on ethical design, regulatory oversight, and responsible platform management. However, the widespread availability of AI tools and persistent demand suggest that completely eradicating misuse is unlikely.
To address this rapidly evolving threat, a multipronged approach is essential:
- Companies must invest in more robust and adaptive safeguards that anticipate jailbreaking attempts.
- Legislators and regulators need to enforce clear standards requiring identity-protecting image generation features be disabled or strictly controlled.
- Social media platforms must improve content moderation and require explicit consent for sharing images featuring real people.
- International cooperation is necessary to manage cross-border challenges posed by AI-generated sexualised content.
This ongoing debate highlights the urgent need to thoughtfully integrate technological capabilities with legal and ethical frameworks to protect individuals from non-consensual exploitation while fostering responsible AI innovation.
Author: Simon Thorne, Senior Lecturer in Computing and Information Systems, Cardiff Metropolitan University
Published: January 13, 2026
Source: The Conversation UK
Full article
