Two countries have blocked the Grok app after it was widely used to generate non-consensual, near-nude deepfakes of women and children. A third country has launched a formal investigation. At the same time, U.S. lawmakers are urging Apple to remove both Grok and X from the App Store over what they call “sickening content generation.”
Grok is available as a standalone app, inside the X app, and through the Grok tab on the X website. Users have shown that it can take normal photos and digitally alter them to remove clothing or replace it with revealing images. Some of the material involved minors. While Grok claims to block explicit nudity, people have used prompts that bypass those limits.
On Friday, U.S. Senators Ron Wyden, Ed Markey, and Ben Ray Luján sent an open letter to the CEOs of Apple and Google. They asked both companies to temporarily remove X and Grok from their app stores “pending a full investigation” into the “mass generation of nonconsensual sexualized images of women and children.” The senators said some of the content may qualify as child sexual abuse material.
They also criticized what they described as a lack of action by Elon Musk and contrasted it with how quickly Apple and Google removed other apps when requested by the government. Musk’s response has been limited to restricting image generation to paid users on X. However, the same tools remain accessible through the Grok tab on the X website and app.
Malaysia and Indonesia block Grok
Associated Press reports that Malaysia and Indonesia have become the first countries to block Grok nationwide.
Regulators in both countries said existing controls failed to stop the creation and spread of fake pornographic content, especially involving women and minors. Indonesia temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday.
Indonesian Communication and Digital Affairs Minister Meutya Hafid said, “The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space.” Officials added that the move aims to protect women, children, and the wider public from AI-generated fake pornography.
Indonesia’s digital supervision authority said early findings showed Grok lacks effective safeguards to prevent the creation and distribution of pornographic content based on real photos. It warned that such misuse violates privacy and image rights and causes psychological, social, and reputational harm.
In Malaysia, the communications regulator ordered a temporary restriction after what it described as “repeated misuse” of Grok to generate obscene, sexually explicit, and non-consensual images, including content involving women and minors. It said earlier notices to X Corp. and xAI relied mainly on user reporting tools. “The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing,” the regulator said.
U.K. opens a formal investigation
Britain’s media regulator, Ofcom, has also opened a formal investigation into whether X complied with its legal duties under the Online Safety Act.
Ofcom said there have been reports of Grok being used to create and share undressed images of people that may amount to intimate image abuse or pornography. It also cited “sexualised images of children that may amount to child sexual abuse material.” Technology Secretary Liz Kendall said, “The content created and shared using Grok in recent days has been deeply disturbing.”
So far, Apple and Google have not issued public responses. Both apps remain available on their U.S. app stores. Requests for comment to xAI drew an automated reply that said, “Legacy Media Lies.”
For now, the situation remains unresolved. Two countries have acted. A third is investigating. U.S. lawmakers are pressing platform owners. The apps, however, continue to operate in major markets as regulators and companies decide what comes next.
