The social media platform X has sparked significant backlash after individuals utilized its AI chatbot Grok to manipulate images of females by taking off their clothing.
Several instances have surfaced where the technology has been used to digitally undress female subjects to show them in bikinis without consent, as well as to put them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the firm responsible for Grok, remained silent to requests for comment, except for an auto-reply message stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to prohibit image stripping tools. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a prison sentence and significant penalties".
"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the risk" of people viewing unlawful material on their networks, but did not specify whether it was at this time examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some additional features—which answers X users' prompts when they reference it in a message.
It is commonly used to offer commentary or additional background to other users' comments, but people on X are also capable of alter an posted photo through its AI image editing tool.
Background of Criticism and Rules
The tool has been faulted for permitting users to generate images and footage with nudity and sexualised content, and it was in the past accused of generating graphic content of celebrities.
"The platform has been enabling the generation and spread of these pictures for a long time without taking any action and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying images of individuals in a pornographic manner".
In a comment, Ofcom said it was unlawful to "make or share non-consensual intimate images or illegal content" and clarified this included sexual deepfakes created with AI.
It said networks such as X were mandated to take "appropriate steps" to "reduce the danger" of users encountering illegal material on their services and to take down it quickly when they become aware of it.
