Malaysia and Indonesia become first countries to block Musk’s Grok over sexualized AI images
Listen to this article
About 4 minutes
The audio version of this article is generated by AI-based technology. Pronunciation errors may occur. We work with our partners to continually review and improve results.
Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk’s company xAI, amid growing concerns among global authorities that it is being misused to generate sexually explicit and non-consensual images.
The moves reflect growing scrutiny of generative AI tools capable of producing realistic images, sounds and text, as well as concerns that existing safeguards are failing to prevent their abuse.
The Grok chatbot, accessible through Musk’s social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or in sexually explicit poses, as well as images involving children.
Last week, Grok limited image generation and editing to paying users following a global backlash against sexualized deepfakes of people, but critics say that hasn’t entirely solved the problem.
An emailed request for comment from The Associated Press to xAI resulted in an automated response from the media support email address reading “Legacy Media Lies.” This was the same message received from another email when asked to comment on the global reaction.
Regulators in the two Southeast Asian countries said existing controls did not prevent the creation and distribution of fake pornographic content, particularly involving women and minors. The Indonesian government temporarily blocked access to Grok on Saturday, followed by Malaysia’s on Sunday.
“The government considers non-consensual sexual deepfakes a serious violation of the human rights, dignity and security of citizens in the digital space,” Indonesian Minister of Communication and Digital Affairs Meutya Hafid said in a statement on Saturday.
The ministry said the move aims to protect women, children and the wider community from fake pornographic content generated using AI.
Initial findings showed that Grok does not have effective safeguards to prevent users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, general director of digital space supervision, said in a separate statement.
He said such practices risk violating privacy and publicity rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm.
In Kuala Lumpur, the Malaysian Communications and Multimedia Commission on Sunday ordered a temporary restriction on Grok, after what it called “repeated misuse” of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors.
The regulator said notices issued this month to X Corp. and xAI demanding stricter safeguards had prompted responses that relied primarily on user reporting mechanisms.
“The restriction is imposed as a preventative and proportionate measure while legal and regulatory processes are ongoing,” the statement said, adding that access will remain blocked until effective safeguards are put in place.
Investigation opened in the United Kingdom
Launching in 2023, Grok is free on X. Users can ask it questions on the social media platform and tag posts they directly created or replies to other users’ posts. Last summer, the company added an image generation feature, Grok Imagine, which included a so-called spicy mode that could generate adult content.
The restrictions in Southeast Asia come as Grok faces increasing scrutiny elsewhere, including in the European Union, the United Kingdom, India and France.
A Toronto mom says her 12-year-old son asked Tesla’s Grok which soccer player was better: Cristiano Ronaldo or Lionel Messi. After some back and forth, she says the chatbot asked her son, “Why don’t you send me nudes?”
On Monday, Britain’s media regulator said it had launched a formal investigation into whether Grok “complied with its obligations to protect citizens of the United Kingdom from illegal content.”
The regulator, Ofcom, said Grok-generated images of sexualised children or undressed people may constitute pornography or child sexual abuse material.
“The content created and shared using Grok in recent days has been deeply troubling,” said Technology Secretary Liz Kendall.
Canada is not considering banning X, AI Minister Evan Solomon said in a post on the platform last week. His office added that discussions about the deepfakes controversy were ongoing.
