The European Union has opened a formal investigation into xAI’s Grok chatbot after reports that it generated large volumes of child sexual abuse material. The move adds to growing pressure on the company as regulators and lawmakers question how the tool was released with weak safety controls.
Grok can generate images from text prompts inside its app, on the web, and through X. Unlike other AI image tools, its guardrails appear loose. That design choice has led to the creation of non-consensual sexual images, including images involving children. Critics say the scale of the problem points to a failure to manage obvious risks.
Earlier this month, three US senators asked Apple CEO Tim Cook to temporarily remove both X and Grok from the App Store. They cited what they described as “sickening content generation.” Apple and Google have not taken action so far. At the same time, authorities in California and the UK have already opened their own investigations, and the two countries have blocked the app.
Scale of the content problem
Engadget reports that one estimate suggests Grok generated around 23,000 child sexual abuse images in just 11 days.
The figures come from research by the Center for Countering Digital Hate, a British nonprofit. It analyzed a random sample of 20,000 images generated by Grok between December 29 and January 9, then extrapolated those results across the full dataset.
According to the group, Grok produced an estimated 3 million sexualized images during that period. Based on their analysis:
- About 23,000 of those images involved children
- The system generated roughly 190 sexualized images per minute
- A sexualized image of a child appeared about once every 41 seconds
EU steps in under digital law
EU investigation falls under the Digital Services Act. Regulators will assess whether xAI took adequate steps to reduce the risks linked to deploying Grok on X and the spread of content that “may amount to child sexual abuse material.”
EU tech chief Henna Virkkunen called non-consensual sexual deepfakes of women and children “a violent, unacceptable form of degradation.” Her statement underscores the political weight behind the probe.
If regulators find that xAI breached the Digital Services Act, the company could face fines of up to 6 percent of its annual global revenue. The outcome could also shape how AI tools are allowed to launch in Europe going forward, especially those tied directly to large social platforms.
