Jakarta, INTI - Indonesia’s Ministry of Communication and Digital Affairs (Kemkomdigi) is currently investigating the alleged misuse of Grok AI, an artificial intelligence feature on platform X, which is suspected of being used to produce and circulate explicit content, including the manipulation of private photos without consent.
Alexander Sabar, Director General of Digital Space Oversight at Kemkomdigi, said preliminary findings indicate that Grok AI lacks clear and adequate safeguard to prevent the creation and distribution of pornographic content based on real photographs of Indonesian citizens.
“Initial findings suggest there are no specific controls within Grok AI to prevent the use of this technology for generating and disseminating pornographic content derived from personal photos. This poses a serious risk of violating individuals’ privacy and image rights,” he started.
The digital manipulation of personal photographs is viewed not only as a violation of public decency, but also as a form of stripping individuals of control over their visual identity. Such practices carry the potential to cause psychological harm, social consequences, and reputational damage to victims.
Coordination Efforts and Potential Sanctions for Platforms
Kemkomdigi is currently coordinating with Electronic System Providers (ESPs) to ensure that all artificial intelligence technologies are equipped with effective privacy protection mechanisms.
These efforts include strengthening content moderation systems, preventing the creation of explicit deepfakes, and establishing rapid response procedures for handling reports of privacy violations and image rights abuse.
“Every Electronic System Provider (ESP) is required to ensure that the technologies they offer are not used to violate privacy, exploit individuals sexually, or undermine a person’s dignity,” emphasized Alexander Sabar.
If non-compliance or uncooperative behavior is identified, Kemkomdigi has the authority to impose administrative sanctions, including the blocking of access to Grok AI and platform X in Indonesia.
Furthermore, service providers or users found to have produced or distributed pornographic content or manipulated personal photos without authorization may face administrative and/or criminal penalties.
Alexander explained that since the enactment of Law No. 1 of 2023 on the Criminal Code (KUHP), effective January 2, 2026, regulations regarding pornography are detailed in Articles 172 and 407.
Article 172 defines pornography as media containing obscenity or sexual exploitation that violates public morality.
Meanwhile, Article 407 stipulates criminal penalties ranging from a minimum of six months to a maximum of ten years in prison, or fines in accordance with the law.
Reporting Mechanisms and Guidance for AI Users
Kemkomdigi also stated that members of the public who fall victim to photo manipulation, sexual deepfakes, or violations of images rights have the option to pursue legal action.
Victims can report such cases to law enforcement authorities or file a complaint with Kemkomdigi.
“We urge everyone to use artificial intelligence technologies responsibly. The digital space is not a lawless area, every individual’s privacy and image rights must be respected and protected,” he emphasized.
Read more: AI Adoption Expected to Surge in 2026