The online network X has caused significant outrage after people utilized its AI chatbot Grok to change photos of women by stripping their garments.
Numerous instances have emerged where the tool has been used to digitally undress female subjects to make them appear in bikinis without permission, as well as to put them in sexually explicit situations.
Company Reaction and Regulatory Position
XAI, the company behind Grok, remained silent to questions, except for an auto-reply response stating "legacy media lies".
A government representative stated that new laws is being brought forward to outlaw nudification technology. Under a new criminal offence, anyone who provided such technology would "receive a prison sentence and substantial fines".
"X or Grok is capable of halting these forms of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "consider the danger" of people accessing unlawful material on their platforms, but did not confirm whether it was presently examining X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' queries when they reference it in a post.
It is often used to provide reactions or more context to other posters' comments, but people on X are also able to alter an posted photo through its AI image editing tool.
Background of Criticism and Rules
The system has been faulted for permitting users to produce photos and footage with explicit content and sexualised content, and it was previously accused of creating inappropriate content of celebrities.
"The platform has been allowing the creation and spread of these pictures for a long time without taking any measures and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "make or circulate private pictures or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said platforms such as X were mandated to take "necessary actions" to "minimize the risk" of users encountering illegal content on their platforms and to take down it promptly when they learn of it.
