The online platform X has caused significant backlash after people employed its AI chatbot Grok to manipulate pictures of women by removing their attire.
Multiple cases have surfaced where the tool has been used to remove clothing from women to show them in swimwear without consent, as well as to place them in sexually explicit contexts.
Developer Reaction and Regulatory Stance
XAI, the firm behind Grok, remained silent to requests for comment, other than an auto-reply message stating "legacy media lies".
A Home Office official stated that new laws is being proposed to outlaw image stripping technology. Under a upcoming law, anyone who provided such technology would "be subject to a jail term and significant penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people viewing unlawful material on their platforms, but did not specify whether it was presently examining X or Grok in relation to AI-generated images.
Grok is a no-cost AI assistant—with some paid-for features—which reacts to X users' requests when they tag it in a post.
It is commonly used to provide reactions or additional background to other posters' remarks, but people on X are also able to modify an uploaded photo through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The system has been condemned for allowing users to create photos and videos with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in generating graphic material of public figures.
"X has been enabling the generation and spread of these images for a long time without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "make or share private pictures or illegal content" and confirmed this included explicit AI images created with AI.
It said platforms such as X were required to take "suitable actions" to "minimize the danger" of users viewing unlawful material on their platforms and to take down it quickly when they are notified of it.
